Skip to content

Run yang validation in db migrator #3102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Nov 13, 2024
Merged

Conversation

ganglyu
Copy link
Contributor

@ganglyu ganglyu commented Dec 27, 2023

What I did

Add unit test/runtime alert for db_migrator.py/load_minigraph: ConfigDB passing yang validation

How I did it

Use subprocess to run yang validation, and then warm-reboot performance will not change.
Check /etc/sonic/mgmt_test_mark, this mark_file means this device is under end to end test.

  • For unit test, db_migrator will wait for validation result and raise exception.
  • For end to end test, db_migrator will wait for validation result and raise exception.

How to verify it

Run unit test and run end to end test

Previous command output (if the output of a command-line utility has changed)

New command output (if the output of a command-line utility has changed)

@ganglyu ganglyu requested a review from qiluo-msft March 21, 2024 13:03
wangxin pushed a commit to sonic-net/sonic-mgmt that referenced this pull request Mar 26, 2024
What is the motivation for this PR?
We need a flag to mark that DUT is under end to end test.
sonic-net/sonic-utilities#3102
db migrator needs to detect DUT is running in end to end test.

How did you do it?
Create /etc/sonic/mgmt_test_mark with fixture.

How did you verify/test it?
Run end to end test, and check /etc/sonic/mgmt_test_mark.
@ganglyu ganglyu closed this Apr 2, 2024
@ganglyu ganglyu reopened this Apr 2, 2024
except sonic_yang.SonicYangException as e:
log.log_error("Yang validation failed: " + str(e))
raise
if len(yang_parser.tablesWithOutYang):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could make it 100% in our StarLab DUT.
However, I suspect if we could cover it 100% in PROD devices. It will break the PROD workflow if not fully coverred in PROD.
I prefer we make the check as log_warning and not raise exception right now.

@ganglyu ganglyu marked this pull request as ready for review October 11, 2024 08:23
@ganglyu ganglyu merged commit 9677447 into sonic-net:master Nov 13, 2024
7 checks passed
@ganglyu ganglyu deleted the yang_validation branch November 13, 2024 03:57
wangxin pushed a commit to sonic-net/sonic-mgmt that referenced this pull request Dec 9, 2024
…n added to yang model (#15947)

What is the motivation for this PR?
In sonic-net/sonic-utilities#3102, the feature comparing yang model with configDB was added to sonic-utilities submodule, but feature subnet decap haven't been added to yang model, so the PR test advancing sonic-utilities to buildimage would fail in subnet decap test.

How did you do it?
Created issue sonic-net/sonic-buildimage#21090 and temporary skip subnet decap test until issue resolved

How did you verify/test it?
qiluo-msft pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Jan 16, 2025
Fixed issue : #20575

Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

How to verify it
Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

Tested branch (Please provide the tested image version)
 202411

Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.
mssonicbld added a commit to mssonicbld/sonic-buildimage that referenced this pull request Jan 17, 2025
issue : sonic-net#20575

<!--
     Please make sure you've read and understood our contributing guidelines:
     https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md

     ** Make sure all your commits include a signature generated with `git commit -s` **

     If this is a bug fix, make sure your description includes "fixes #xxxx", or
     "closes #xxxx" or "resolves #xxxx"

     Please provide the following information:
-->

#### Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

##### Work item tracking
- Microsoft ADO **(number only)**:

#### How I did it

#### How to verify it

Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->

#### Which release branch to backport (provide reason below if selected)

<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->

- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
- [ ] 202211
- [ ] 202305

#### Tested branch (Please provide the tested image version)

<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->

- [x] 202411

#### Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.

<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->

<!--
 Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->

#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->

#### A picture of a cute animal (not mandatory but encouraged)
VladimirKuk pushed a commit to Marvell-switching/sonic-buildimage that referenced this pull request Jan 21, 2025
Fixed issue : sonic-net#20575

Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

How to verify it
Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

Tested branch (Please provide the tested image version)
 202411

Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.
StormLiangMS pushed a commit to sonic-net/sonic-buildimage that referenced this pull request Jan 23, 2025
issue : #20575

<!--
     Please make sure you've read and understood our contributing guidelines:
     https://github.com/Azure/SONiC/blob/gh-pages/CONTRIBUTING.md

     ** Make sure all your commits include a signature generated with `git commit -s` **

     If this is a bug fix, make sure your description includes "fixes #xxxx", or
     "closes #xxxx" or "resolves #xxxx"

     Please provide the following information:
-->

#### Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

##### Work item tracking
- Microsoft ADO **(number only)**:

#### How I did it

#### How to verify it

Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

<!--
If PR needs to be backported, then the PR must be tested against the base branch and the earliest backport release branch and provide tested image version on these two branches. For example, if the PR is requested for master, 202211 and 202012, then the requester needs to provide test results on master and 202012.
-->

#### Which release branch to backport (provide reason below if selected)

<!--
- Note we only backport fixes to a release branch, *not* features!
- Please also provide a reason for the backporting below.
- e.g.
- [x] 202006
-->

- [ ] 201811
- [ ] 201911
- [ ] 202006
- [ ] 202012
- [ ] 202106
- [ ] 202111
- [ ] 202205
- [ ] 202211
- [ ] 202305

#### Tested branch (Please provide the tested image version)

<!--
- Please provide tested image version
- e.g.
- [x] 20201231.100
-->

- [x] 202411

#### Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.

<!--
Write a short (one line) summary that describes the changes in this
pull request for inclusion in the changelog:
-->

<!--
 Ensure to add label/tag for the feature raised. example - PR#2174 under sonic-utilities repo. where, Generic Config and Update feature has been labelled as GCU.
-->

#### Link to config_db schema for YANG module changes
<!--
Provide a link to config_db schema for the table for which YANG model
is defined
Link should point to correct section on https://github.com/Azure/sonic-buildimage/blob/master/src/sonic-yang-models/doc/Configuration.md
-->

#### A picture of a cute animal (not mandatory but encouraged)
prabhataravind pushed a commit to prabhataravind/sonic-buildimage that referenced this pull request Mar 5, 2025
Fixed issue : sonic-net#20575

Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

How to verify it
Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

Tested branch (Please provide the tested image version)
 202411

Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.
yanjundeng pushed a commit to yanjundeng/sonic-buildimage that referenced this pull request Apr 23, 2025
Fixed issue : sonic-net#20575

Why I did it
"config-reload" in dualtor topologies were failing due to absence of TC_TO_DSCP Yang model.
The above failure was seen after the the PR sonic-net/sonic-utilities#3102

How to verify it
Step-1: In DUT add the yang file to "/usr/local/yang-models/sonic-tc-dscp-map.yang" to this path.
Step-2: config reload -y

Tested branch (Please provide the tested image version)
 202411

Description for the changelog
Adding YANG model for TC_TO_DSCP_MAP along with test files.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants