[advanced-reboot] Set timeout for reboot test to prevent hung tests #5365
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
5 tasks
wangxin
approved these changes
Mar 22, 2022
wangxin
pushed a commit
that referenced
this pull request
Mar 25, 2022
…5365) Advanced-reboot testcases sometimes gets hung inside ptf. This was earlier handled as part of #4532 Although PR 4532 sets a timeout for PTF test, the inbuilt --test-case-timeout interrupts the advanced-reboot test as expected, but sometimes that is not enough. After the timeout interrupt, the test still proceeds to collect logs and analyze the pcap files. In an unlikely event when the analysis logic is hung, the whole testcase stays hung indefinitely. This commit fixes this testcase hung issue.
xwjiang-ms
pushed a commit
to xwjiang-ms/sonic-mgmt
that referenced
this pull request
Apr 13, 2022
…onic-net#5365) Advanced-reboot testcases sometimes gets hung inside ptf. This was earlier handled as part of sonic-net#4532 Although PR 4532 sets a timeout for PTF test, the inbuilt --test-case-timeout interrupts the advanced-reboot test as expected, but sometimes that is not enough. After the timeout interrupt, the test still proceeds to collect logs and analyze the pcap files. In an unlikely event when the analysis logic is hung, the whole testcase stays hung indefinitely. This commit fixes this testcase hung issue.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
Summary: Run advanced-reboot PTF test using a thread which can be timedout and killed if the test gets hung.
Fixes # (issue)
Type of change
Back port request
Approach
What is the motivation for this PR?
Advanced-reboot testcases sometimes gets hung inside ptf.
This was earlier handled as part of #4532
Although PR 4532 sets a timeout for PTF test, the inbuilt
--test-case-timeoutinterrupts the advanced-reboot test as expected, but sometimes that is not enough.After the timeout interrupt, the test still proceeds to collect logs and analyze the pcap files.
In a unlikely event when the analysis logic is hung, the whole testcase stays hung indefinitely.
This PR fixes this testcase hung issue.
How did you do it?
Start a PTF test inside a thread, and timeout the thread and kill the test inside ptf container.
How did you verify/test it?
Executed on a physical testbed.
Any platform specific information?
Supported testbed topology if it's a new test case?
Documentation