Skip to content

[fdborch] Fixed Orchagent crash in FDB flush on port disable. #1369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 29, 2020
Merged

[fdborch] Fixed Orchagent crash in FDB flush on port disable. #1369

merged 2 commits into from
Jul 29, 2020

Conversation

rajkumar38
Copy link
Contributor

@rajkumar38 rajkumar38 commented Jul 29, 2020

Signed-off-by: Rajkumar Pennadam Ramamoorthy [email protected]

What I did
Fixed Orchagent crash in FDB flush on port disable.
Steps to reproduce:

  1. configure vlan and a port.
  2. Learn MAC and populate FDB table.
  3. Disable the port.

In fdborch.cpp:485, "m_entries" element is erased (from function "storeFdbEntryState") inside range based loop.
Iterators or references to erased element are invalidated. Hence update the iterator before removing the element.

Why I did it

How I verified it
Validate the above testcase with fix.

Details if related
Logs:

Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Core was generated by `/usr/bin/orchagent -d /var/log/swss -b 8192 -m 00:50:b6:50:51:51'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0 0xb6b5676c in std::local_Rb_tree_increment (__x=0x3633240a, __x@entry=0x69b2e0) at ../../../../../src/libstdc++-v3/src/c++98/tree.cc:65
65 ../../../../../src/libstdc++-v3/src/c++98/tree.cc: No such file or directory.
[Current thread is 1 (Thread 0xb6f69000 (LWP 44))]
(gdb) bt
#0 0xb6b5676c in std::local_Rb_tree_increment (__x=0x3633240a, __x@entry=0x69b2e0) at ../../../../../src/libstdc++-v3/src/c++98/tree.cc:65
#1 std::_Rb_tree_increment (__x=__x@entry=0x69b2e0) at ../../../../../src/libstdc++-v3/src/c++98/tree.cc:91
#2 0x004bfea0 in std::_Rb_tree_const_iterator::operator++ (this=) at /usr/include/c++/6/bits/stl_tree.h:288
#3 FdbOrch::update (this=this@entry=0x5ace68, type=, entry=, bridge_port_id=16325548649219474) at fdborch.cpp:235
#4 0x004c0746 in FdbOrch::doTask (this=0x5ace68, consumer=...) at fdborch.cpp:485
#5 0x004684a6 in OrchDaemon::start (this=0x59b57c) at orchdaemon.cpp:467
#6 0x00458e10 in main (argc=, argv=) at main.cpp:346
(gdb) p * __x
Cannot access memory at address 0x3633240a
(gdb) p __x
$1 = (std::_Rb_tree_node_base ) 0x3633240a
(gdb) fr 1
#1 std::_Rb_tree_increment (__x=__x@entry=0x69b2e0) at ../../../../../src/libstdc++-v3/src/c++98/tree.cc:91
91 in ../../../../../src/libstdc++-v3/src/c++98/tree.cc
(gdb) p __x
$2 = (const std::_Rb_tree_node_base ) 0x69b2e0
(gdb) p
__x
$3 = {_M_color = (unknown: 11347552), _M_parent = 0xa0d3324, _M_left = 0xd4c4544, _M_right = 0x3633240a}
(gdb) p __x->_M_right
$4 = (std::_Rb_tree_node_base::_Base_ptr) 0x3633240a
(gdb) p
__x->_M_right
Cannot access memory at address 0x3633240a
(gdb) p/x m_entries
$1 = std::set with 4159 elements = {[0] = {mac = {m_mac = {0x0, 0x0, 0x0, 0xaa, 0x11, 0x12}}, bv_id = 0x26000000000591, port_name = "Ethernet0"}, [1] = {

@rajkumar38
Copy link
Contributor Author

@vasant17 @lguohan Pls review

@@ -232,13 +232,14 @@ void FdbOrch::update(sai_fdb_event_t type,
update.entry.mac.to_string().c_str(),
vlanName.c_str(), update.port.m_alias.c_str());

for (const auto& itr : m_entries)
for (auto itr = m_entries.begin(); itr != m_entries.end();)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.
Just a suggestion, can we just update the for loop to the below, instead of increment iterator at multiple places?

for (auto itr = m_entries.begin(); itr != m_entries.end(); itr = next_item)
{
    auto next_item = std::next(itr, 1);
    .....
    ....
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoided multiple iterator increment. Updated patch. Pls check.

Fixed review comments.
 Signed-off-by: Rajkumar Pennadam Ramamoorthy <[email protected]>
@prsunny prsunny merged commit 26e1723 into sonic-net:master Jul 29, 2020
@rajkumar38 rajkumar38 deleted the fdb_bug branch July 30, 2020 04:01
EdenGri pushed a commit to EdenGri/sonic-swss that referenced this pull request Feb 28, 2022
**- What I did**
Add information related to updating firmware image from sonic-installer.
fix sonic-net/sonic-buildimage#6264

**- How I did it**
Replace click.echo with a function that combines echo and logger functions

**- How to verify it**
Start the sonic-installer process and then grep syslog
$ sonic-installer install http://not_exist.com
$ tail -n 100 /var/log/syslog | grep "sonic-installer"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants