Skip to content

[warm-reboot]: added automated recover for ISSU file #1466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Mar 11, 2021

Conversation

vadymhlushko-mlnx
Copy link
Contributor

Signed-off-by: Vadym Hlushko [email protected]

What I did

Modified pre-check for file /host/warmboot/issu_bank.txt in warm-reboot script.
Added automated recovery of /host/warmboot/issu_bank.txt

How I did it

Modified the warm-reboot script.

How to verify it

Need to somehow corrupt the /host/warmboot/issu_bank.txt (list below) and then run the warm-reboot command.
For testing I have modified the warm-reboot script - added corruption command before check_issu_bank_file function call.

  1. Remove issu_bank.txt
  2. Clear issu_bank.txt
  3. Change characters count in issu_bank.txt
  4. Write invalid content into issu_bank.txt

Previous command output (if the output of a command-line utility has changed)

root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
To recover (/host/warmboot/issu_bank.txt) file, do the following:
$ docker exec -it syncd sx_api_dbg_generate_dump.py
$ docker exec -it syncd cat /tmp/sdkdump | grep 'ISSU Bank'
Command above will print the VALUE of ISSU BANK - 0 or 1, use this VALUE in the next command
$ printf VALUE > /host/warmboot/issu_bank.txt

New command output (if the output of a command-line utility has changed)

root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
Recovering the (/host/warmboot/issu_bank.txt) file

@vadymhlushko-mlnx
Copy link
Contributor Author

@qiluo-msft, @yxieca, @stepanblyschak could you please review?

Copy link
Contributor

@qiluo-msft qiluo-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As comment

yxieca
yxieca previously approved these changes Mar 1, 2021
@qiluo-msft
Copy link
Contributor

qiluo-msft commented Mar 2, 2021

    check_issu_bank_file "$BEFORE_PRE_SHUTDOWN"

Not used, you can remove the argument, and also remove the variable. #Closed


Refers to: scripts/fast-reboot:583 in 1fd720f. [](commit_id = 1fd720f, deletion_comment = False)

Copy link
Contributor

@qiluo-msft qiluo-msft left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unblock, minor issue

@qiluo-msft qiluo-msft self-requested a review March 2, 2021 05:32
@qiluo-msft qiluo-msft dismissed their stale review March 2, 2021 05:32

Unblock

@vadymhlushko-mlnx
Copy link
Contributor Author

    check_issu_bank_file "$BEFORE_PRE_SHUTDOWN"

Not used, you can remove the argument, and also remove the variable.

Refers to: scripts/fast-reboot:583 in 1fd720f. [](commit_id = 1fd720f, deletion_comment = False)

Done

@dprital
Copy link
Collaborator

dprital commented Mar 10, 2021

@qiluo-msft - Who can merge it ? we need it also for 202012. Thanks.

@qiluo-msft qiluo-msft merged commit e1c52a0 into sonic-net:master Mar 11, 2021
daall pushed a commit that referenced this pull request Mar 12, 2021
#### What I did
Modified pre-check for file `/host/warmboot/issu_bank.txt` in `warm-reboot` script.
Added automated recovery of  `/host/warmboot/issu_bank.txt`

#### How I did it
Modified the `warm-reboot` script.

#### How to verify it
Need to somehow corrupt the `/host/warmboot/issu_bank.txt` (list below) and then run the `warm-reboot` command.
For testing I have modified the `warm-reboot` script - added corruption command before `check_issu_bank_file` function call.

1. Remove `issu_bank.txt`
2. Clear `issu_bank.txt`
3. Change characters count in `issu_bank.txt`
4. Write invalid content into `issu_bank.txt`

#### Previous command output (if the output of a command-line utility has changed)
```
root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
To recover (/host/warmboot/issu_bank.txt) file, do the following:
$ docker exec -it syncd sx_api_dbg_generate_dump.py
$ docker exec -it syncd cat /tmp/sdkdump | grep 'ISSU Bank'
Command above will print the VALUE of ISSU BANK - 0 or 1, use this VALUE in the next command
$ printf VALUE > /host/warmboot/issu_bank.txt
```

#### New command output (if the output of a command-line utility has changed)
```
root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
Recovering the (/host/warmboot/issu_bank.txt) file
```
@prsunny
Copy link
Contributor

prsunny commented May 11, 2022

@vaibhavhd for viz

abdosi pushed a commit that referenced this pull request May 17, 2022
#### What I did
Modified pre-check for file `/host/warmboot/issu_bank.txt` in `warm-reboot` script.
Added automated recovery of  `/host/warmboot/issu_bank.txt`

#### How I did it
Modified the `warm-reboot` script.

#### How to verify it
Need to somehow corrupt the `/host/warmboot/issu_bank.txt` (list below) and then run the `warm-reboot` command.
For testing I have modified the `warm-reboot` script - added corruption command before `check_issu_bank_file` function call.

1. Remove `issu_bank.txt`
2. Clear `issu_bank.txt`
3. Change characters count in `issu_bank.txt`
4. Write invalid content into `issu_bank.txt`

#### Previous command output (if the output of a command-line utility has changed)
```
root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
To recover (/host/warmboot/issu_bank.txt) file, do the following:
$ docker exec -it syncd sx_api_dbg_generate_dump.py
$ docker exec -it syncd cat /tmp/sdkdump | grep 'ISSU Bank'
Command above will print the VALUE of ISSU BANK - 0 or 1, use this VALUE in the next command
$ printf VALUE > /host/warmboot/issu_bank.txt
```

#### New command output (if the output of a command-line utility has changed)
```
root@arc-switch1041:~# warm-reboot
(/host/warmboot/issu_bank.txt) does NOT exist or empty ...
Recovering the (/host/warmboot/issu_bank.txt) file
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants