Skip to content

Update test_po_cleanup.py #5453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 4, 2022
Merged

Update test_po_cleanup.py #5453

merged 3 commits into from
Apr 4, 2022

Conversation

wenyiz2021
Copy link
Contributor

@wenyiz2021 wenyiz2021 commented Mar 31, 2022

Description of PR

For a multi-asic device, restart of swss and teamd is done per asic, noticed that for a 6-asic device, asic3 and asic4 are experiencing more time during restart. Also, interfaces and bgp neighbors need more minutes to come back. Execute a config_reload at the end of test_po_cleanup helps to bring back processes in a safe manner.
duthost.asic_instance(enum_asic_index).start_service("swss")
assert wait_until(300, 20, 0, duthost.critical_services_fully_started),\

      "Not all critical services are fully started"

E AssertionError: Not all critical services are fully started
duthost =
duthosts = []
enum_asic_index = 3
enum_rand_one_per_hwsku_frontend_hostname = 'xxx'
tbinfo = {'auto_recover': 'True', 'comment': 'Arvind', 'conf-name': 'xxx', 'duts': ['xxx'], ...}
[pc/test_po_cleanup.py:66: AssertionError]

  1. Execute an "config_reload" at the end of test_po_cleanup, to start up not only all critical services, but also wait for all port, interfaces, bgp neighbors to come up.
  2. Call duthost.stop_service("swss") directly from duthost, instead of passing each asics inside the test_po_cleanup. to avoid execution sequence of test_po_cleanup[asic0], test_po_cleanup_after_reload[duthost], test_po_cleanup[asic1].

Summary:
Fixes # (issue)
#5294
#4887

Type of change

  • Bug fix
  • Testbed and Framework(new/improvement)
  • [ X] Test case(new/improvement)

Back port request

  • [X ] 201911
  • 202012

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

pc/test_po_cleanup.py::test_po_cleanup[str-masic-acs-2] PASSED [100%]

---------------------------------------------------------------------------------------------- generated xml file: /var/src/sonic-mgmt-int/tests/logs/tr.xml ----------------------------------------------------------------------------------------------
=============================================================================================================== 1 passed in 489.35 seconds ================================================================================================================

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

[test_po_cleanup] Change timeout length to 800 instead of 300 to allow more time for multi-asic asics to come back.
@wenyiz2021 wenyiz2021 requested a review from judyjoseph March 31, 2022 22:54
@wenyiz2021 wenyiz2021 requested a review from a team as a code owner March 31, 2022 22:54
@wenyiz2021
Copy link
Contributor Author

Same issue as: #5294

@wenyiz2021
Copy link
Contributor Author

pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-0] PASSED [ 14%]
pc/test_po_cleanup.py::test_po_cleanup_after_reload[str-multi-asic-acs-2] PASSED [ 28%]
pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-1] PASSED [ 42%]
pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-2] PASSED [ 57%]
pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-3] PASSED [ 71%]
pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-4] PASSED [ 85%]
pc/test_po_cleanup.py::test_po_cleanup[str-multi-asic-acs-2-5] PASSED [100%]

============================================================================================================== 7 passed in 1630.24 seconds

1. Execute an "config_reload" at the end of test_po_cleanup, to start up not only all critical services, but also wait for all port, interfaces, bgp neighbors to come up.
2. Call duthost.stop_service("swss") directly from duthost, instead of passing each asics inside the test_po_cleanup. to avoid execution sequence of test_po_cleanup[asic0], test_po_cleanup_after_reload[duthost], test_po_cleanup[asic1].
@wenyiz2021
Copy link
Contributor Author

New change commited:
pc/test_po_cleanup.py::test_po_cleanup[str-masic-acs-2] PASSED [100%]

---------------------------------------------------------------------------------------------- generated xml file: /var/src/sonic-mgmt-int/tests/logs/tr.xml ----------------------------------------------------------------------------------------------
=============================================================================================================== 1 passed in 489.35 seconds ================================================================================================================

@wenyiz2021
Copy link
Contributor Author

New change description:

  1. Execute an "config_reload" at the end of test_po_cleanup, to start up not only all critical services, but also wait for all port, interfaces, bgp neighbors to come up.
  2. Call duthost.stop_service("swss") directly from duthost, instead of passing each asics inside the test_po_cleanup. to avoid execution sequence of test_po_cleanup[asic0], test_po_cleanup_after_reload[duthost], test_po_cleanup[asic1].

@wenyiz2021 wenyiz2021 merged commit f0737b9 into sonic-net:202012 Apr 4, 2022
@judyjoseph
Copy link
Contributor

@wenyiz2021 Can you port this PR to master branch also ?

wenyiz2021 added a commit that referenced this pull request Apr 15, 2022
* Update test_po_cleanup.py

Update same changes in master branch from PR: (#5453)
wenyiz2021 added a commit that referenced this pull request Apr 22, 2022
* Update test_po_cleanup.py

Update same changes in master branch from PR: (#5453)

* Update test_fan_drawer_fans.py

* Update test_fan_drawer_fans.py

* Update test_fan_drawer_fans.py

Disable thermalctld when running test_fan_drawer_fans.py
wenyiz2021 added a commit that referenced this pull request Apr 22, 2022
* Update test_po_cleanup.py

Update same changes in master branch from PR: (#5453)

* Update test_fan_drawer_fans.py

* Update test_fan_drawer_fans.py

* Update test_fan_drawer_fans.py

Disable thermalctld when running test_fan_drawer_fans.py

(cherry picked from commit 35f25e1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants