[chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9075

wenyiz2021 · 2023-07-21T01:54:46Z

Description of PR

There are 2 fixes in this PR to enhance for chassis:

Right now in processes_utils.py when check_critical_services, we rely on what is pre-defined in inventory file, otherwise it is 300 sec, but for SUP, we should give 900sec as most SUPs have 10+ dockers that need to come back up, it should differ from linecards or any single-dut. magic 900 sec comes from experientment, where I had run twice the test, 1st time need 800+sec, 2nd time needs ~702sec
During memory exhaustion test, if the random selected dut is chassis SUP, we also need to check linecards bootup and critical services are up.

Summary:
Fixes # (issue)
#9087

Type of change

Bug fix
Testbed and Framework(new/improvement)
Test case(new/improvement)

Back port request

201911
202012
202205

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

mssonicbld · 2023-07-21T01:55:47Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/platform_tests/test_memory_exhaustion.py:51:21: F821 undefined name 'check_interfaces_and_services' tests/platform_tests/test_memory_exhaustion.py:51:55: F821 undefined name 'interfaces' tests/platform_tests/test_memory_exhaustion.py:51:67: F821 undefined name 'xcvr_skip_list' tests/platform_tests/test_memory_exhaustion.py:52:63: F821 undefined name 'reboot_type' check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld · 2023-07-21T01:59:59Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed - hook id: trailing-whitespace - exit code: 1 - files were modified by this hook Fixing tests/platform_tests/test_memory_exhaustion.py fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/platform_tests/test_memory_exhaustion.py:53:79: F821 undefined name 'reboot_type' check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld · 2023-07-21T03:09:15Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Failed - hook id: trailing-whitespace - exit code: 1 - files were modified by this hook Fixing tests/platform_tests/test_memory_exhaustion.py fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Passed check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

mssonicbld · 2023-07-21T18:26:53Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/platform_tests/test_memory_exhaustion.py:54:5: E303 too many blank lines (2) tests/platform_tests/test_memory_exhaustion.py:85:51: F821 undefined name 'conn_graph_facts' check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

kenneth-arista · 2023-07-26T17:46:24Z

#9087

arlakshm · 2023-07-26T21:40:50Z

tests/platform_tests/test_memory_exhaustion.py

@@ -24,7 +24,8 @@ class TestMemoryExhaustion:
    """

    @pytest.fixture(autouse=True)
-    def tearDown(self, duthosts, enum_rand_one_per_hwsku_hostname, localhost, pdu_controller):
+    def tearDown(self, duthosts, enum_rand_one_per_hwsku_hostname,
+                 localhost, pdu_controller,):


remove extra comma at end #Closed

also fix the line indentation

removed the comma.
Indentation is correct I believe:

sonic-mgmt/tests/platform_tests/api/test_sfp.py

Line 37 in 9d6c0de

xcvr_skip_list, conn_graph_facts, shutdown_ebgp): # noqa F811

arlakshm · 2023-07-26T21:48:44Z

tests/platform_tests/test_memory_exhaustion.py

+            # For sup, we also need to ensure linecards are back and healthy for following tests
+            is_sup = duthost.get_facts().get("modular_chassis") and duthost.is_supervisor_node()
+            if is_sup:
+                for lc in duthosts.frontend_nodes:
+                    wait_for_startup(lc, localhost, delay=10, timeout=300)
+                    wait_critical_processes(lc)


can we write commmon function for this code, it is being used couple time here may be useful in other tests as well

mssonicbld · 2023-07-27T16:48:58Z

The pre-commit check detected issues in the files touched by this pull request.
The pre-commit check is a mandatory check, please fix detected issues.

Detailed pre-commit check results:
trim trailing whitespace.................................................Passed fix end of files.........................................................Passed check yaml...........................................(no files to check)Skipped check for added large files..............................................Passed check python ast.........................................................Passed flake8...................................................................Failed - hook id: flake8 - exit code: 1 tests/platform_tests/test_memory_exhaustion.py:30:38: F821 undefined name 'localhost' tests/platform_tests/test_memory_exhaustion.py:52:13: F821 undefined name 'wait_lc_healthy_if_sup' tests/platform_tests/test_memory_exhaustion.py:80:9: F821 undefined name 'wait_lc_healthy_if_sup' check conditional mark sort..........................(no files to check)Skipped

To run the pre-commit checks locally, you can follow below steps:

Ensure that default python is python3. In sonic-mgmt docker container, default python is python2. You can run
the check by activating the python3 virtual environment in sonic-mgmt docker container or outside of sonic-mgmt
docker container.
Ensure that the pre-commit package is installed:

sudo pip install pre-commit

Go to repository root folder
Install the pre-commit hooks:

pre-commit install

Use pre-commit to check staged file:

pre-commit

Alternatively, you can check committed files using:

pre-commit run --from-ref <commit_id> --to-ref <commit_id>

tests/platform_tests/test_memory_exhaustion.py

arlakshm · 2023-07-31T20:56:57Z

class TestMemoryExhaustion:

same as above

Refers to: tests/platform_tests/test_memory_exhaustion.py:21 in 8505fe6. [](commit_id = 8505fe6, deletion_comment = False)

…is sup (sonic-net#9075)

mssonicbld · 2023-08-04T08:05:21Z

Cherry-pick PR to 202205: #9295

…is sup (#9075)

…is sup (sonic-net#9075)

mssonicbld · 2023-08-08T06:13:36Z

Cherry-pick PR to 202305: #9329

…is sup (#9075)

…is sup (sonic-net#9075)

Update test_memory_exhaustion.py

77d2074

Update test_memory_exhaustion.py

27c7eab

Update test_memory_exhaustion.py

430b8d6

wenyiz2021 added 3 commits July 21, 2023 10:58

fix pre-commit

9f61aa7

Update processes_utils.py

322b678

Update test_memory_exhaustion.py

9f55b7d

Update test_memory_exhaustion.py

53a3b38

wenyiz2021 changed the title ~~Update test_memory_exhaustion.py~~ [chassis] accomodate test_memory_exhaustion.py if rand due is chassis sup Jul 21, 2023

Update test_memory_exhaustion.py

0ba2f2f

wenyiz2021 requested a review from arlakshm July 21, 2023 18:49

wenyiz2021 marked this pull request as ready for review July 21, 2023 18:49

wenyiz2021 requested a review from prgeor as a code owner July 21, 2023 18:49

wenyiz2021 changed the title ~~[chassis] accomodate test_memory_exhaustion.py if rand due is chassis sup~~ [chassis] accomodate test_memory_exhaustion.py if random due is chassis sup Jul 21, 2023

wenyiz2021 added chassis Request for 202205 branch labels Jul 21, 2023

Update processes_utils.py

d5d1df2

wenyiz2021 changed the title ~~[chassis] accomodate test_memory_exhaustion.py if random due is chassis sup~~ [chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup Jul 21, 2023

rlhui assigned wenyiz2021 Jul 26, 2023

arlakshm reviewed Jul 26, 2023

View reviewed changes

Update test_memory_exhaustion.py

8505fe6

wenyiz2021 requested a review from arlakshm July 31, 2023 20:31

arlakshm reviewed Jul 31, 2023

View reviewed changes

tests/platform_tests/test_memory_exhaustion.py Outdated Show resolved Hide resolved

arlakshm reviewed Jul 31, 2023

View reviewed changes

tests/platform_tests/test_memory_exhaustion.py Outdated Show resolved Hide resolved

wenyiz2021 merged commit d94858c into sonic-net:master Aug 3, 2023

wenyiz2021 deleted the memory_exhaustion branch August 3, 2023 01:08

wangxin added Approved for 202205 branch and removed Approved for 202205 branch labels Aug 4, 2023

mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Aug 4, 2023

[chassis] accomodate test_memory_exhaustion.py if random dut is chass…

2728190

…is sup (sonic-net#9075)

mssonicbld added the Created PR to 202205 branch label Aug 4, 2023

mssonicbld mentioned this pull request Aug 4, 2023

[action] [PR:9075] [chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9295

Merged

6 tasks

mssonicbld pushed a commit that referenced this pull request Aug 4, 2023

[chassis] accomodate test_memory_exhaustion.py if random dut is chass…

cd3b341

…is sup (#9075)

mssonicbld added Included in 202205 branch and removed Created PR to 202205 branch labels Aug 4, 2023

wenyiz2021 mentioned this pull request Aug 7, 2023

[regression] fix regression in test_memory_exhaustion.py #9320

Merged

6 tasks

lizhijianrd added Request for 202305 branch Approved for 202305 branch labels Aug 8, 2023

mssonicbld pushed a commit to mssonicbld/sonic-mgmt that referenced this pull request Aug 8, 2023

[chassis] accomodate test_memory_exhaustion.py if random dut is chass…

b1c88ce

…is sup (sonic-net#9075)

mssonicbld added the Created PR to 202305 branch label Aug 8, 2023

This was referenced Aug 8, 2023

[action] [PR:9075] [chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9329

Merged

[action] [PR:9320] [regression] fix regression in test_memory_exhaustion.py #9330

Merged

mssonicbld pushed a commit that referenced this pull request Aug 8, 2023

[chassis] accomodate test_memory_exhaustion.py if random dut is chass…

a08b96e

…is sup (#9075)

mssonicbld added Included in 202305 branch and removed Created PR to 202305 branch labels Aug 8, 2023

mssonicbld mentioned this pull request Aug 8, 2023

[action] [PR:9320] [regression] fix regression in test_memory_exhaustion.py #9339

Merged

6 tasks

AharonMalkin pushed a commit to AharonMalkin/sonic-mgmt that referenced this pull request Jan 25, 2024

[chassis] accomodate test_memory_exhaustion.py if random dut is chass…

945d089

…is sup (sonic-net#9075)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9075

[chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9075

Uh oh!

wenyiz2021 commented Jul 21, 2023 •

edited by wangxin

Loading

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

kenneth-arista commented Jul 26, 2023

Uh oh!

arlakshm Jul 26, 2023 •

edited

Loading

Uh oh!

arlakshm Jul 26, 2023

Uh oh!

wenyiz2021 Jul 27, 2023

Uh oh!

arlakshm Jul 26, 2023

Uh oh!

mssonicbld commented Jul 27, 2023

Uh oh!

Uh oh!

Uh oh!

arlakshm commented Jul 31, 2023

Uh oh!

mssonicbld commented Aug 4, 2023

Uh oh!

mssonicbld commented Aug 8, 2023

Uh oh!

Uh oh!

[chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9075

[chassis] accomodate test_memory_exhaustion.py if random dut is chassis sup #9075

Uh oh!

Conversation

wenyiz2021 commented Jul 21, 2023 • edited by wangxin Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of PR

Type of change

Back port request

Approach

What is the motivation for this PR?

How did you do it?

How did you verify/test it?

Any platform specific information?

Supported testbed topology if it's a new test case?

Documentation

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

mssonicbld commented Jul 21, 2023

Uh oh!

kenneth-arista commented Jul 26, 2023

Uh oh!

arlakshm Jul 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arlakshm Jul 26, 2023

Choose a reason for hiding this comment

Uh oh!

wenyiz2021 Jul 27, 2023

Choose a reason for hiding this comment

Uh oh!

arlakshm Jul 26, 2023

Choose a reason for hiding this comment

Uh oh!

mssonicbld commented Jul 27, 2023

Uh oh!

Uh oh!

Uh oh!

arlakshm commented Jul 31, 2023

Uh oh!

mssonicbld commented Aug 4, 2023

Uh oh!

mssonicbld commented Aug 8, 2023

Uh oh!

Uh oh!

wenyiz2021 commented Jul 21, 2023 •

edited by wangxin

Loading

arlakshm Jul 26, 2023 •

edited

Loading