Buffer and qos j2 file modularization #21102

zhixzhu · 2024-12-10T00:44:06Z

Why I did it

It usually requires update on both upstream j2 files and platform j2 files, when there is new configuration.
This change is intend to reduce changes in upstream j2 files in future.

Work item tracking

Microsoft ADO (number only):

How I did it

For each config section, define a macro for it.

How to verify it

Verified with T0/T1/T2 config.

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Zhixin Zhu <[email protected]>

mssonicbld · 2024-12-19T05:55:19Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2024-12-19T05:55:32Z

Azure Pipelines successfully started running 1 pipeline(s).

kevinskwang · 2024-12-23T07:01:05Z

files/build_templates/qos_config.j2

@@ -106,6 +115,9 @@
        }
    },
 {% endif %}
+{% if (generate_pfc_to_queue_map is defined) %}


Any case we need to define generate_pfc_to_queue_map rather than use the default 1-1 PFC to queue mapping?

No specific override is defined. A placeholder for future.

kevinskwang · 2024-12-23T07:05:27Z

files/build_templates/buffers_config.j2

@@ -82,7 +87,12 @@ def
    {%- if cable_len -%}
        {{ cable_len.0 }}
    {%- else %}
-        {%- if 'torrouter' in switch_role.lower() and 'mgmt' not in switch_role.lower()%}


why we swap the if/else branch here?

This is to handle the scenario where a device (including tor router) may have backplane ports.
In such a case, set the cable length of backplane port before the tor router check.

kevinskwang · 2024-12-23T07:09:41Z

files/build_templates/buffers_config.j2

@@ -188,7 +190,16 @@ def
    {%- if port_names_list_inactive.append(port) %}{%- endif %}
 {%- endfor %}
 {%- set port_names_inactive  = port_names_list_inactive  | join(',') %}
+
+{%- set port_names_list_all  = [] %}


Shall we move code line 194-198 under the if branch {% if (defs.generate_cable_length_config is defined) %} at line 200?

yes, we can move under the if

Signed-off-by: Zhixin Zhu <[email protected]>

mssonicbld · 2025-01-02T22:43:37Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-01-02T22:43:51Z

Azure Pipelines successfully started running 1 pipeline(s).

alpeshspatel · 2025-01-03T16:39:02Z

@kevinwangsk FYI

zhixzhu · 2025-01-14T18:27:57Z

#21056 has similar change in qos_config.j2.

Signed-off-by: Zhixin Zhu <[email protected]>

mssonicbld · 2025-01-28T18:47:27Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-01-28T18:47:39Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-01-28T19:12:40Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-01-28T19:12:52Z

Azure Pipelines successfully started running 1 pipeline(s).

alpeshspatel · 2025-02-11T12:52:15Z

@kevinskwang @kperumalbfn FYI.

alpeshspatel · 2025-02-28T16:17:11Z

@kevinskwang can you redo the /azp run Azure.sonic-buildimage

kevinskwang · 2025-03-04T18:04:44Z

/azp run

azure-pipelines · 2025-03-04T18:04:50Z

Commenter does not have sufficient privileges for PR 21102 in repo sonic-net/sonic-buildimage

zhixzhu · 2025-03-06T18:56:51Z

For the failure in "Azure.sonic-buildimage (Test kvmtest-t1-lag by Elastictest)", couldn't access the log: https://dev.azure.com/mssonic/build/_build/results?buildId=758211

kevinskwang · 2025-03-14T09:55:03Z

@kperumalbfn could you help to review this?

kevinskwang · 2025-03-14T09:56:29Z

/azp run

azure-pipelines · 2025-03-14T09:56:35Z

Commenter does not have sufficient privileges for PR 21102 in repo sonic-net/sonic-buildimage

kperumalbfn · 2025-03-14T23:09:25Z

files/build_templates/buffers_config.j2

@@ -82,7 +87,12 @@ def
    {%- if cable_len -%}
        {{ cable_len.0 }}
    {%- else %}
-        {%- if 'torrouter' in switch_role.lower() and 'mgmt' not in switch_role.lower()%}
+        {%- if port_name.startswith('Ethernet-BP') %}


@alpeshspatel Are these internal ports?

Yes, Ethernet-BP are internal ports.

kperumalbfn · 2025-03-14T23:10:33Z

files/build_templates/buffers_config.j2

@@ -82,7 +87,12 @@ def
    {%- if cable_len -%}
        {{ cable_len.0 }}
    {%- else %}
-        {%- if 'torrouter' in switch_role.lower() and 'mgmt' not in switch_role.lower()%}
+        {%- if port_name.startswith('Ethernet-BP') %}
+            {%- if 'internal' not in ports2cable %}


Is this only for T2?

Ethernet-BP is mostly for T2.
Some T0/T1 devices also have Ethernet-BP interfaces.

Ethernet-BP is also for some smartswitches too, @kperumalbfn

kperumalbfn · 2025-03-14T23:12:41Z

files/build_templates/qos_config.j2

+{%- set LOSSLESS_TC = [] %}
+{%- if generate_lossless_tc_list is defined %}
+    {{- generate_lossless_tc_list(LOSSLESS_TC) }}
+{%- else %}


Thanks @alpeshspatel, this is good. If we need to have different set of LOSSLESS_TCs, we could use this macro inside SKUs.

Yes Kumaresh - that is the intent

alpeshspatel · 2025-03-16T18:59:08Z

/azp run

azure-pipelines · 2025-03-16T18:59:15Z

Commenter does not have sufficient privileges for PR 21102 in repo sonic-net/sonic-buildimage

alpeshspatel · 2025-03-16T19:00:59Z

@kevinwangsk @kperumalbfn
Can you help with these two failures:

[Azure.sonic-buildimage](https://github.com/sonic-net/sonic-buildimage/pull/21102/checks?check_run_id=36323861335)
Azure.sonic-buildimageFailing after 281m — Build #build_Azure.sonic-buildimage_merge_20250128.20 failed
Required
[Azure.sonic-buildimage (Test kvmtest-t1-lag by Elastictest)](https://github.com/sonic-net/sonic-buildimage/pull/21102/checks?check_run_id=36322227333)
Azure.sonic-buildimage (Test kvmtest-t1-lag by Elastictest)Failing after 103m — Test kvmtest-t1-lag by Elastictest failed

Cannot see the logs as the run is pretty old (Jan 28)

kevinskwang · 2025-03-18T01:13:25Z

/azp run

azure-pipelines · 2025-03-18T01:13:31Z

Commenter does not have sufficient privileges for PR 21102 in repo sonic-net/sonic-buildimage

zhixzhu · 2025-03-18T04:29:19Z

Commenter does not have sufficient privileges for PR 21102 in repo sonic-net/sonic-buildimage

@kevinskwang "/azp run" failed.

kperumalbfn · 2025-03-18T04:45:28Z

/azp run

azure-pipelines · 2025-03-18T04:45:34Z

You have several pipelines (over 10) configured to build pull requests in this repository. Specify which pipelines you would like to run by using /azp run [pipelines] command. You can specify multiple pipelines using a comma separated list.

kperumalbfn · 2025-03-18T04:45:56Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-18T04:46:07Z

Azure Pipelines successfully started running 1 pipeline(s).

zhixzhu · 2025-03-18T16:45:13Z

kvmtest-t0-sonic failed at snmpwalk timeout, not related with buffer/qos j2 files.

test case: macsec/test_interop_protocol.py::TestInteropProtocol::test_snmp[128_SCI]
invocation = {'module_args': {'_raw_params': 'docker exec snmp snmpwalk -v 2c -c public 10.1.0.32 .1.3.6.1.2.1.1.1.0', '_uses_shell': False, 'warn': False, 'stdin_add_newline': True, 'strip_empty_ends': True, 'argv': None, 'chdir': None, 'executable': None, 'creates': None, 'removes': None, 'stdin': None}}
_ansible_no_log = None
stdout =
stderr =
Timeout: No Response from 10.1.0.32

https://elastictest.org/scheduler/publictestplan/67d9148414a0c3680c255abe

kperumalbfn · 2025-03-19T17:02:41Z

files/build_templates/buffers_config.j2

-        {%- if 'torrouter' in switch_role.lower() and 'mgmt' not in switch_role.lower()%}
+        {%- if port_name.startswith('Ethernet-BP') %}
+            {%- if 'internal' not in ports2cable %}
+                {%- set _ = ports2cable.update({'internal': '5m'}) %}


@zhixzhu @alpeshspatel Does internal port support lossless configs?

@zhixzhu @alpeshspatel How about other QoS table mappings for internal ports?

@kperumalbfn Internal ports (back plane ports) support lossless configs. Other qos table mappings are also applied to back plane ports, by adding back plane ports to PORT_ACTIVE list.

alpeshspatel · 2025-03-27T19:39:29Z

@kperumalbfn @kevinskwang should this have a 202411 label?

alpeshspatel · 2025-03-31T04:46:37Z

Hi @kperumalbfn @kevinskwang should this have a 202411 label?

stephenxs · 2025-04-03T00:33:23Z

Hi @kperumalbfn @kevinskwang @r12f
will this PR go to 202412?

kperumalbfn · 2025-04-03T00:36:43Z

Thanks @alpeshspatel @stephenxs added 202412 tag.

mssonicbld · 2025-04-15T19:49:06Z

Cherry-pick PR to msft-202412: Azure/sonic-buildimage-msft#994

Buffer and qos j2 file modularization

zhixzhu added 2 commits December 9, 2024 16:42

qos j2 file modularization

38dafb5

Signed-off-by: Zhixin Zhu <[email protected]>

buffers j2 file modularization

f8bd037

Signed-off-by: Zhixin Zhu <[email protected]>

zhixzhu marked this pull request as ready for review December 19, 2024 05:53

zhixzhu requested a review from lguohan as a code owner December 19, 2024 05:53

zhixzhu changed the title ~~qos j2 file modularization~~ Buffer and qos j2 file modularization Dec 19, 2024

kevinskwang reviewed Dec 23, 2024

View reviewed changes

move variable definition inside if block

02aaf5c

Signed-off-by: Zhixin Zhu <[email protected]>

merge master

507fe8f

Signed-off-by: Zhixin Zhu <[email protected]>

reduce one layer of if-else

27f175f

alpeshspatel approved these changes Jan 28, 2025

View reviewed changes

kevinskwang approved these changes Feb 21, 2025

View reviewed changes

kperumalbfn reviewed Mar 14, 2025

View reviewed changes

kperumalbfn reviewed Mar 19, 2025

View reviewed changes

kperumalbfn approved these changes Mar 19, 2025

View reviewed changes

kperumalbfn enabled auto-merge (squash) March 21, 2025 21:53

kperumalbfn merged commit e6dc8dc into sonic-net:master Mar 21, 2025
22 checks passed

zhixzhu deleted the buffer_qos_j2_modularization branch March 21, 2025 23:36

kperumalbfn added the Request for msft-202412 Branch label Apr 3, 2025

r12f added the Approved for msft-202412 Branch label Apr 15, 2025

mssonicbld mentioned this pull request Apr 15, 2025

[action] [PR:21102] Buffer and qos j2 file modularization Azure/sonic-buildimage-msft#994

Merged

11 tasks

mssonicbld added the Created PR to msft-202412 Branch label Apr 15, 2025

mssonicbld added Included in msft-202412 Branch and removed Created PR to msft-202412 Branch labels Apr 16, 2025

yanjundeng pushed a commit to yanjundeng/sonic-buildimage that referenced this pull request Apr 23, 2025

Buffer and qos j2 file modularization (sonic-net#21102)

ca78d18

Buffer and qos j2 file modularization

Buffer and qos j2 file modularization #21102

Buffer and qos j2 file modularization #21102

Conversation

zhixzhu commented Dec 10, 2024 • edited Loading

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

mssonicbld commented Dec 19, 2024

azure-pipelines bot commented Dec 19, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alpeshspatel Dec 23, 2024 • edited Loading

Choose a reason for hiding this comment

mssonicbld commented Jan 2, 2025

azure-pipelines bot commented Jan 2, 2025

alpeshspatel commented Jan 3, 2025

zhixzhu commented Jan 14, 2025

mssonicbld commented Jan 28, 2025

azure-pipelines bot commented Jan 28, 2025

mssonicbld commented Jan 28, 2025

azure-pipelines bot commented Jan 28, 2025

alpeshspatel commented Feb 11, 2025

alpeshspatel commented Feb 28, 2025

kevinskwang commented Mar 4, 2025

azure-pipelines bot commented Mar 4, 2025

zhixzhu commented Mar 6, 2025

kevinskwang commented Mar 14, 2025

kevinskwang commented Mar 14, 2025

azure-pipelines bot commented Mar 14, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alpeshspatel commented Mar 16, 2025

azure-pipelines bot commented Mar 16, 2025

alpeshspatel commented Mar 16, 2025

kevinskwang commented Mar 18, 2025

azure-pipelines bot commented Mar 18, 2025

zhixzhu commented Mar 18, 2025

kperumalbfn commented Mar 18, 2025

azure-pipelines bot commented Mar 18, 2025

kperumalbfn commented Mar 18, 2025

azure-pipelines bot commented Mar 18, 2025

zhixzhu commented Mar 18, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alpeshspatel commented Mar 27, 2025

alpeshspatel commented Mar 31, 2025

stephenxs commented Apr 3, 2025

kperumalbfn commented Apr 3, 2025

mssonicbld commented Apr 15, 2025

zhixzhu commented Dec 10, 2024 •

edited

Loading

alpeshspatel Dec 23, 2024 •

edited

Loading