Skip to content

[Chassis][voq] remote link down ECMP acceleration #3150

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 23 commits into from
May 30, 2024
Merged

[Chassis][voq] remote link down ECMP acceleration #3150

merged 23 commits into from
May 30, 2024

Conversation

arlakshm
Copy link
Contributor

@arlakshm arlakshm commented May 14, 2024 •
edited
Loading

What I did
Handle update of nexthops from nexthop group when the port on the remote linecards goes up or down
MSFT ADO 28218126
Why I did it
Handing of remote port state change faster before the protocol converges.

How I verified it
The following changes are done

  • new oper_status field is added to the SYSTEM_INTERFACE entry. When the port state changes then the local linecard will update oper_status of SYSTEM_INTERFACE.
  • The remote linecard handles the SYSTEM_INTERFACE state change and removes the nexthops this port belongs to from ECMP groups

Details if related

Testing logs

   
May 28 18:50:27.899388 str2-sonic-lc3-1 WARNING swss#orchagent: :- doTask: Handle remote systemport intf str2-sonic-lc5-1|asic0|PortChannel104, oper status down

May 28 18:50:27.899434 str2-sonic-lc3-1 NOTICE swss#orchagent: :- ifChangeInformRemoteNextHop: Inband port Ethernet-IB0

May 28 18:50:27.899434 str2-sonic-lc3-1 NOTICE swss#orchagent: :- ifChangeInformRemoteNextHop: Found remote Neighbor 10.0.0.5 on str2-sonic-lc5-1|asic0|PortChannel104  
May 28 18:50:27.899434 str2-sonic-lc3-1 NOTICE swss#orchagent: :- ifChangeInformRemoteNextHop: Found remote NextHop 10.0.0.5 on Ethernet-IB0  

^^ The nh removed ecmp group
May 28 18:50:27.899453 str2-sonic-lc3-1 NOTICE swss#orchagent: :- setNextHopFlag: setNextHopFlag on 10.0.0.5 seen on port Ethernet-IB0

May 28 18:50:27.901375 str2-sonic-lc3-1 NOTICE swss#orchagent: :- ifChangeInformRemoteNextHop: Found remote Neighbor fc00::a on str2-sonic-lc5-1|asic0|PortChannel104   
May 28 18:50:27.901375 str2-sonic-lc3-1 NOTICE swss#orchagent: :- ifChangeInformRemoteNextHop: Found remote NextHop fc00::a on Ethernet-IB0
 May 28 18:50:27.901375 str2-sonic-lc3-1 NOTICE swss#orchagent: :- setNextHopFlag: setNextHopFlag on fc00::a seen on port Ethernet-IB0  --> The nh removed ecmp group
May 28 18:50:27.902803 str2-sonic-lc3-1 NOTICE swss#orchagent: :- setIntf: Set interface str2-sonic-lc5-1|asic0|PortChannel104, vrf_id 3000000000034, ip_prefix NULL, adminUp 0, mtu 0, loopbackAction   
May 28 18:50:28.097430 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel1019
May 28 18:50:28.097430 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel101
 May 28 18:50:28.097430 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel101
May 28 18:50:28.097449 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel103
 May 28 18:50:28.097449 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel103
 May 28 18:50:28.097465 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel105
 May 28 18:50:28.097512 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel105
 May 28 18:50:28.097512 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel107
 May 28 18:50:28.097512 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel107
 May 28 18:50:28.097512 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel109
 May 28 18:50:28.097534 str2-sonic-lc3-1 INFO swss#orchagent: :- doLagMemberTask: Failed to locate LAG str2-sonic-lc3-1|Asic0|PortChannel109
 May 28 18:50:28.097534 str2-sonic-lc3-1 INFO swss#orchagent: :- doTask: Port eth1-midplane doesn't exist
 May 28 18:50:28.141178 str2-sonic-lc3-1 NOTICE swss#orchagent: :- removeNeighbor: Failed to remove still referenced neighbor 0a:b3:76:05:02:a7 on str2-sonic-lc5-1|asic0|PortChannel104
  May 28 18:50:28.263249 str2-sonic-lc3-1 NOTICE swss#orchagent: :- addNextHopGroup: Create next hop group 10.0.0.9@Ethernet-IB0,10.0.0.17@Ethernet-IB0,10.0.0.21@Ethernet-IB0,10.0.0.29@Ethernet-IB0,10.0.0.33@Ethernet-IB0,10.0.0.35@Ethernet-IB0,10.0.0.49@Ethernet-IB0,10.0.0.51@Ether\
net-IB0,10.0.0.53@Ethernet-IB0,10.0.0.55@Ethernet-IB0
May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 1, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 2, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 3, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 4, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 5, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 6, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 7, 3, 0
 May 28 18:50:28.263249 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 8, 3, 0
 May 28 18:50:28.263320 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 9, 3, 0
 May 28 18:50:28.263320 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 10, 3, 0
May 28 18:50:28.270507 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_creating_entries: ObjectBulker.flush creating_entries 10
May 28 18:50:28.270562 str2-sonic-lc3-1 INFO swss#orchagent: :- remove_entry: EntityBulker.remove_entry 1, 1
May 28 18:50:28.271635 str2-sonic-lc3-1 NOTICE swss#orchagent: :- addNextHopGroup: Create next hop group fc00::12@Ethernet-IB0,fc00::22@Ethernet-IB0,fc00::2a@Ethernet-IB0,fc00::3a@Ethernet-IB0,fc00::42@Ethernet-IB0,fc00::46@Ethernet-IB0,fc00::62@Ethernet-IB0,fc00::66@Ethernet-IB0\
,fc00::6a@Ethernet-IB0,fc00::6e@Ethernet-IB0
May 28 18:50:28.271635 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 1, 3, 0
 May 28 18:50:28.271635 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 2, 3, 0
 May 28 18:50:28.271635 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 3, 3, 0
 May 28 18:50:28.271635 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 4, 3, 0
 May 28 18:50:28.271635 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 5, 3, 0
 May 28 18:50:28.271688 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 6, 3, 0
 May 28 18:50:28.271688 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 7, 3, 0
 May 28 18:50:28.271688 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 8, 3, 0
 May 28 18:50:28.271688 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 9, 3, 0
 May 28 18:50:28.271688 str2-sonic-lc3-1 INFO swss#orchagent: :- create_entry: ObjectBulker.create_entry 10, 3, 0
May 28 18:50:28.278666 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_creating_entries: ObjectBulker.flush creating_entries 10
 May 28 18:50:28.278721 str2-sonic-lc3-1 INFO swss#orchagent: :- remove_entry: EntityBulker.remove_entry 2, 1
May 28 18:50:28.279656 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_removing_entries: EntityBulker.flush removing_entries 2
May 28 18:50:28.290457 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_setting_entries: EntityBulker.flush setting_entries, count 4
  May 28 18:50:28.290457 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_setting_entries: EntityBulker.flush setting_entries status[0]=0(0x0x55b208e8ed50)
 May 28 18:50:28.290457 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_setting_entries: EntityBulker.flush setting_entries status[1]=0(0x0x55b208e8ed54)
 May 28 18:50:28.290457 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_setting_entries: EntityBulker.flush setting_entries status[2]=0(0x0x55b20ba74de0)
 May 28 18:50:28.290457 str2-sonic-lc3-1 INFO swss#orchagent: :- flush_setting_entries: EntityBulker.flush setting_entries status[3]=0(0x0x55b20ba74de4)
 May 28 18:50:28.290623 str2-sonic-lc3-1 INFO swss#orchagent: :- addRoutePost: Post set route 0.0.0.0/0 with next hop(s) 10.0.0.9@Ethernet-IB0,10.0.0.17@Ethernet-IB0,10.0.0.21@Ethernet-IB0,10.0.0.29@Ethernet-IB0,10.0.0.33@Ethernet-IB0,10.0.0.35@Ethernet-IB0,10.0.0.49@Ethernet-IB0,\
10.0.0.51@Ethernet-IB0,10.0.0.53@Ethernet-IB0,10.0.0.55@Ethernet-IB0 
^^route removed
May 28 18:50:28.290773 str2-sonic-lc3-1 INFO swss#orchagent: :- removeRoutePost: Remove route 10.0.0.4/31 with next hop(s) 3.3.3.2@Ethernet-IB0

@arlakshm arlakshm changed the title Chassis remote link down EMCP acceleration Chassis remote link down ECMP acceleration May 22, 2024
@arlakshm arlakshm marked this pull request as ready for review May 22, 2024 16:47
@prsunny prsunny requested a review from prabhataravind May 22, 2024 16:56
@arlakshm arlakshm changed the title Chassis remote link down ECMP acceleration [Chassis][voq] remote link down ECMP acceleration May 22, 2024
@deepak-singhal0408
Copy link
Contributor

Can you add the test results? what is the perf improvement we are seeing with this change.

@arlakshm
Copy link
Contributor Author

Can you add the test results? what is the perf improvement we are seeing with this change.

@deepak-singhal0408, updated the PR description with the testing logs. This shows the next hop group is updated before the route is removed.

deepak-singhal0408
deepak-singhal0408 approved these changes May 28, 2024
View reviewed changes
Copy link
Contributor

@deepak-singhal0408 deepak-singhal0408 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.. Thank you.

prsunny
prsunny reviewed May 28, 2024
View reviewed changes
@arlakshm
Copy link
Contributor Author

/Azp run Azure.sonic-swss

@azure-pipelines Azure Pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@prsunny prsunny merged commit 7c7cece into sonic-net:master May 30, 2024
17 checks passed
@arlakshm arlakshm deleted the arlakshm/master/remote_ecmp branch June 4, 2024 20:12
@saksarav-nokia saksarav-nokia mentioned this pull request Jun 20, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@prsunny prsunny prsunny approved these changes

@deepak-singhal0408 deepak-singhal0408 deepak-singhal0408 approved these changes

@abdosi abdosi Awaiting requested review from abdosi abdosi is a code owner

@judyjoseph judyjoseph Awaiting requested review from judyjoseph judyjoseph is a code owner

@prabhataravind prabhataravind Awaiting requested review from prabhataravind

Assignees
No one assigned
Labels
chassis
Projects
SONiC Chassis
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@arlakshm @deepak-singhal0408 @prsunny