Skip to content

SVCPLAN-8877: Add logic to return codes in case of backup failures#16

Open
inwho wants to merge 1 commit intomainfrom
hwang104/SVCPLAN-8877/expand-return-code-for-failed-runs
Open

SVCPLAN-8877: Add logic to return codes in case of backup failures#16
inwho wants to merge 1 commit intomainfrom
hwang104/SVCPLAN-8877/expand-return-code-for-failed-runs

Conversation

@inwho
Copy link

@inwho inwho commented Mar 3, 2026

In cases where backup jobs terminate before completion (runtime error/timeout etc), the resultant files (/var/log/backup_last_run and /var/log/backup_check_last_run) do not report a job status code, which then lead to empty fields in $backup_return_code, $prune_return_code, and $return_code.

This change checks to see if the variables are empty (which implies a failed job run) and if they are, has it report an error code 1.

Consideration: maybe it should return a different error code to signal that the backup job script itself failed?

Script tested locally on cc-adm01.

@inwho inwho requested a review from billglick March 3, 2026 18:12
@inwho inwho self-assigned this Mar 3, 2026
Copy link
Member

@billglick billglick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While your changes fix the issue with incorrect return codes, I still think we need to also figure out why the return codes are failing.

@inwho
Copy link
Author

inwho commented Mar 4, 2026

I've been digging into it and unfortunately can't find a common factor across clusters yet.
Currently working on mg-adm01 which seems to relate to borg backup failures.

cc-adm01 is a bit odd since it sometimes works sometimes doesn't. I wonder if it is related to the recent instability issues.

Copy link
Member

@billglick billglick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good.

I also could not figure out why it was sometimes failing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants