[chassis][dnx] Ports don't come up after reboot/config load_minigraph if we have qos buffer profile/cfg files on a brcm dnx linecard in a T2 chassis

[sanmalho-git]

Description

On a DNX multi-asic linecard in a T2 chassis, the ports don't become operationally up if we have the QoS buffer profile / configuration files added to the device data folder, and:

• config load_minigraph and config save
• Reboot with config_db with qos buffer profile/configuration

Steps to reproduce the issue:

1. For Nokia-IXR7250E-36x400G linecard, add the QoS profile/configuration files to /usr/share/sonic/device/x86_64-nokia_ixr7250e_36x400g-r0/Nokia-IXR7250E-36x400G/0 and /usr/share/sonic/device/x86_64-nokia_ixr7250e_36x400g-r0/Nokia-IXR7250E-36x400G/1 directory
2. Do config load_minigraph and config save
   • All ports stay operationally up.
3. Do config load_minigraph again
   • Ports don't come up
4. Reboot the linecard
   • Ports don't come up

Some triaging showed that the port admin state up message is not sent to SAI and thus ports are not coming up. Actually we don’t see any QOS configs except BUFFER_POOL table creation message sent to SAI.

Workaround is to wait for 180 seconds and do a 'config qos reload' to bring the ports up again.

Describe the results you received:

Port don't come up

Describe the results you expected:

Ports should come up

Output of show version:

(paste your output here)

Output of show techsupport:

(paste your output here or download and attach the file here )

Additional information you deem important (e.g. issue happens only occasionally):

[sanmalho-git]

@judyjoseph @rlhui - Here is the issue regarding 'config qos reload' required after 180 seconds for ports to become operationally up.

Could you please add the chassis label to it - and as discussed in our meeting, assign it to somebody in MSFT to take a look.

[rlhui]

@vmittal-msft - would you please check? Thanks.

[rlhui]

@vmittal-msft - any update on this, thanks

[vmittal-msft]

@rlhui I tried this multiple times on a chassis with LC having single DNX asic. I haven't seen this issue. My QoS/MMU settings are little bit different than what we have in master. Not sure if this makes any difference. I will raise a PR to push these settings soon.

[sanmalho-git]

@vmittal-msft - please let us know the PR# for the differences to the QoS/MMU settings that you have as compared to master. We can try it out on our linecards in Nokia.

[vmittal-msft]

Raised this PR for updated settings -
#11108

[sanmalho-git]

Issue was because of undefined BUFFER_POOL in BUFFER_PROFILE in local copy of the qos buffer profile files. Fixing this (inline with those in PR#11108) fixed the issue.