Skip to content

Occasionally interface event is lost #164

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yue-fred-gao opened this issue Feb 25, 2025 · 0 comments
Open

Occasionally interface event is lost #164

yue-fred-gao opened this issue Feb 25, 2025 · 0 comments

Comments

@yue-fred-gao
Copy link
Contributor

This is seen once but it is worth to record the analysis.

  1. Below message is seen in the log: syncd vl_api_sw_interface_event_t_handler:632: vpp cannot get interface name for sw index 25, which means interface is not found in interface_name_by_sw_index.
  2. When an interface is up, vpp sends the interface-up event to the client (SONiC), which will be stored in a message queue. If it happens that refresh_interfaces_list is called before the event is picked up by vpp event polling thread (SwitchStateBase::vppProcessEvents is the handler), a race condition will happen causing this problem. Here are the details:
  • the interface_name_by_sw_index will be cleared first in api_sw_interface_dump
  • sending SW_INTERFACE_DUMP and PING API requests to vpp to get interface dump
  • calling W to process the responses from the message queue. If the interface up event is already received and currently stored in the queue, vl_api_sw_interface_event_t_handler callback will be invoked when processing the event. Because interface_name_by_sw_index is already cleared, we will end up the error message "vpp cannot get interface name".
  • Potential solution: call vpp_sync_for_events first in refresh_interfaces_list. This will process all the events in the queue. Ultimate solution is changing to some async model, which requires some investigation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant