Gazebo GUI segfault on exit - Plugin::Update called after plugin removed

[srmainwaring]

Environment

• OS Version: macOS Big Sur 11.6.1
• Source or binary build? source build, dev branch forked off main (https://github.com/srmainwaring/ign-gazebo/tree/feature/ign-gazebo7-metal)

Description

Expected behaviour

The GUI should shut down cleanly on exit (either Ctrl-C from terminal, exit button, or Quit in the GUI menu)

Actual behavior:

• In a terminal a segmentation fault occurs.
• When debugging a bad access error is observed.

Steps to reproduce

1. Start an ignition gazebo server session
2. Start an ignition gazebo gui session that includes the EntityTree
3. Exit the gui

The nature of the error message / segfault seems to depend on what plugins are loaded and may differ when stepping through the debugger compared to running in a terminal. Including the EntityTree plugin will always result in a terminal segfault on exit, although the issue can observed in a debugger with just a minimal set of GUI plugins (MinimalScene, GzSceneManager, InteractiveViewControl and CameraTracking).

From what I can tell it appears that the update thread continues to call the Plugin::Update method after the plugin has been removed in the application destructor ignition::gui::~Application().

Could there be a problem here:

https://github.com/ignitionrobotics/ign-gazebo/blob/97f2e9c8499fdeb89d45220a89b7d570c9ce6690/src/gui/GuiRunner.cc#L265-L275

If the plugin is unloaded after the plugins list has been populated, but before plugin->Update() has been called, then the pointer would be invalid.

There is also some odd behaviour in the main thread stack trace (for instance ~Application() is called 3x)?

I appreciate that I am observing this on development branch on an unsupported platform, so wouldn't usually bother the team, but I don't think the shutdown sequence is particular to macOS and I'd like to try and resolve the issue before submitting a PR for macOS Metal support based on this branch.

Output

Terminal output

Stack trace (most recent call last):
#26   Object "ruby", at 0x1068c1f09, in main + 99
#25   Object "libruby.3.0.dylib", at 0x10693bdbb, in ruby_run_node + 86
#24   Object "libruby.3.0.dylib", at 0x10693bf27, in rb_ec_exec_node + 288
#23   Object "libruby.3.0.dylib", at 0x106a7adac, in rb_vm_exec + 1907
#22   Object "libruby.3.0.dylib", at 0x106a6bb49, in vm_exec_core + 7454
#21   Object "libruby.3.0.dylib", at 0x106a7eb32, in vm_sendish + 1168
#20   Object "libruby.3.0.dylib", at 0x106a84913, in vm_call_cfunc_with_frame + 345
#19   Object "fiddle.bundle", at 0x10810b716, in function_call + 1486
#18   Object "libruby.3.0.dylib", at 0x106a46eef, in rb_nogvl + 178
#17   Object "fiddle.bundle", at 0x10810bd0e, in nogvl_ffi_call + 26
#16   Object "libffi.dylib", at 0x7fff2dac022c, in ffi_call_int + 698
#15   Object "libffi.dylib", at 0x7fff2dac08f5, in ffi_call_unix64 + 85
#14   Object "libignition-gazebo7-ign.7.0.0~pre1.", at 0x108153ced, in runGui + 45
#13   Object "libignition-gazebo7-gui.7.0.0~pre1.", at 0x10821ca8d, in ignition::gazebo::v7::gui::runGui(int&, char**, char const*, char const*) + 301
#12   Object "libignition-gui7.7.0.0~pre1.dylib", at 0x108e1939e, in ignition::gui::Application::~Application() + 14
#11   Object "libignition-gui7.7.0.0~pre1.dylib", at 0x108e18d68, in ignition::gui::Application::~Application() + 1240
#10   Object "QtWidgets", at 0x10ba4c00e, in QApplication::~QApplication() + 830
#9    Object "QtCore", at 0x10c7edb24, in QObject::~QObject() + 1924
#8    Object "QtCore", at 0x10c7eddd1, in QObjectPrivate::deleteChildren() + 289
#7    Object "libignition-gazebo7-gui.7.0.0~pre1.", at 0x108226a6e, in ignition::gazebo::v7::GuiRunner::~GuiRunner() + 14
#6    Object "libignition-gazebo7-gui.7.0.0~pre1.", at 0x108226a0f, in ignition::gazebo::v7::GuiRunner::~GuiRunner() + 79
#5    Object "libignition-gazebo7-gui.7.0.0~pre1.", at 0x1082783d3, in void ignition::utils::detail::DefaultDelete<ignition::gazebo::v7::GuiRunner::Implementation>(ignition::gazebo::v7::GuiRunner::Implementation*) + 67
#4    Object "libignition-gazebo7.7.0.0~pre1.dyli", at 0x108543fdd, in ignition::gazebo::v7::EntityComponentManager::~EntityComponentManager() + 29
#3    Object "libignition-gazebo7.7.0.0~pre1.dyli", at 0x10855883d, in ignition::gazebo::EntityComponentManagerPrivate::~EntityComponentManagerPrivate() + 1421
#2    Object "libsystem_platform.dylib", at 0x7fff20525d7c, in _sigtramp + 28
#1    Object "libignition-tools-backward.dylib", at 0x10811cfad, in backward::SignalHandling::sig_handler(int, __siginfo*, void*) + 13
#0    Object "libignition-tools-backward.dylib", at 0x10811d016, in backward::SignalHandling::handleSignal(int, __siginfo*, void*) + 70
zsh: segmentation fault  ign gazebo -v4 -g

Debugging message and stack trace

Exception has occurred. EXC_BAD_ACCESS (code=1, address=0x10)

Main thread:

std::__1::__vector_base<sdf::v13::Sensor, std::__1::allocator<sdf::v13::Sensor> >::~__vector_base() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/vector:463)
std::__1::vector<sdf::v13::Sensor, std::__1::allocator<sdf::v13::Sensor> >::~vector() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/vector:557)
std::__1::vector<sdf::v13::Sensor, std::__1::allocator<sdf::v13::Sensor> >::~vector() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/vector:552)
sdf::v13::Joint::Implementation::~Implementation() (/Users/rhys/Code/osrf/ign_fortress_ws/src/sdformat/src/Joint.cc:34)
sdf::v13::Joint::Implementation::~Implementation() (/Users/rhys/Code/osrf/ign_fortress_ws/src/sdformat/src/Joint.cc:34)
void ignition::utils::detail::DefaultDelete<sdf::v13::Joint::Implementation>(sdf::v13::Joint::Implementation*) (/usr/local/include/ignition/utils1/ignition/utils/detail/DefaultOps.hh:48)
std::__1::unique_ptr<sdf::v13::Joint::Implementation, void (*)(sdf::v13::Joint::Implementation*)>::reset(sdf::v13::Joint::Implementation*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1689)
std::__1::unique_ptr<sdf::v13::Joint::Implementation, void (*)(sdf::v13::Joint::Implementation*)>::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
std::__1::unique_ptr<sdf::v13::Joint::Implementation, void (*)(sdf::v13::Joint::Implementation*)>::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
ignition::utils::ImplPtr<sdf::v13::Joint::Implementation, void (*)(sdf::v13::Joint::Implementation*), ignition::utils::detail::CopyMoveDeleteOperations<sdf::v13::Joint::Implementation, sdf::v13::Joint::Implementation* (*)(sdf::v13::Joint::Implementation const&), void (*)(sdf::v13::Joint::Implementation&, sdf::v13::Joint::Implementation const&)> >::~ImplPtr() (/usr/local/include/ignition/utils1/ignition/utils/ImplPtr.hh:105)
ignition::utils::ImplPtr<sdf::v13::Joint::Implementation, void (*)(sdf::v13::Joint::Implementation*), ignition::utils::detail::CopyMoveDeleteOperations<sdf::v13::Joint::Implementation, sdf::v13::Joint::Implementation* (*)(sdf::v13::Joint::Implementation const&), void (*)(sdf::v13::Joint::Implementation&, sdf::v13::Joint::Implementation const&)> >::~ImplPtr() (/usr/local/include/ignition/utils1/ignition/utils/ImplPtr.hh:105)
sdf::v13::Joint::~Joint() (/usr/local/include/sdformat-13.0/sdf/Joint.hh:85)
sdf::v13::Joint::~Joint() (/usr/local/include/sdformat-13.0/sdf/Joint.hh:85)
std::__1::pair<unsigned long long const, sdf::v13::Joint>::~pair() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/utility:297)
std::__1::pair<unsigned long long const, sdf::v13::Joint>::~pair() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/utility:297)
void std::__1::allocator_traits<std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, void*> > >::destroy<std::__1::pair<unsigned long long const, sdf::v13::Joint>, void, void>(std::__1::allocator<std::__1::__tree_node<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, void*> >&, std::__1::pair<unsigned long long const, sdf::v13::Joint>*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/__memory/allocator_traits.h:317)
std::__1::__tree<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::__map_value_compare<unsigned long long, std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::less<unsigned long long>, true>, std::__1::allocator<std::__1::__value_type<unsigned long long, sdf::v13::Joint> > >::destroy(std::__1::__tree_node<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, void*>*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/__tree:1801)
std::__1::__tree<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::__map_value_compare<unsigned long long, std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::less<unsigned long long>, true>, std::__1::allocator<std::__1::__value_type<unsigned long long, sdf::v13::Joint> > >::destroy(std::__1::__tree_node<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, void*>*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/__tree:1799)
std::__1::__tree<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::__map_value_compare<unsigned long long, std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::less<unsigned long long>, true>, std::__1::allocator<std::__1::__value_type<unsigned long long, sdf::v13::Joint> > >::~__tree() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/__tree:1789)
std::__1::__tree<std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::__map_value_compare<unsigned long long, std::__1::__value_type<unsigned long long, sdf::v13::Joint>, std::__1::less<unsigned long long>, true>, std::__1::allocator<std::__1::__value_type<unsigned long long, sdf::v13::Joint> > >::~__tree() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/__tree:1786)
std::__1::map<unsigned long long, sdf::v13::Joint, std::__1::less<unsigned long long>, std::__1::allocator<std::__1::pair<unsigned long long const, sdf::v13::Joint> > >::~map() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/map:1092)
std::__1::map<unsigned long long, sdf::v13::Joint, std::__1::less<unsigned long long>, std::__1::allocator<std::__1::pair<unsigned long long const, sdf::v13::Joint> > >::~map() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/map:1090)
ignition::gazebo::RenderUtilPrivate::~RenderUtilPrivate() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/rendering/RenderUtil.cc:101)
ignition::gazebo::RenderUtilPrivate::~RenderUtilPrivate() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/rendering/RenderUtil.cc:101)
std::__1::default_delete<ignition::gazebo::RenderUtilPrivate>::operator()(ignition::gazebo::RenderUtilPrivate*) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1428)
std::__1::unique_ptr<ignition::gazebo::RenderUtilPrivate, std::__1::default_delete<ignition::gazebo::RenderUtilPrivate> >::reset(ignition::gazebo::RenderUtilPrivate*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1689)
std::__1::unique_ptr<ignition::gazebo::RenderUtilPrivate, std::__1::default_delete<ignition::gazebo::RenderUtilPrivate> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
std::__1::unique_ptr<ignition::gazebo::RenderUtilPrivate, std::__1::default_delete<ignition::gazebo::RenderUtilPrivate> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
ignition::gazebo::v7::RenderUtil::~RenderUtil() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/rendering/RenderUtil.cc:539)
ignition::gazebo::v7::RenderUtil::~RenderUtil() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/rendering/RenderUtil.cc:538)
ignition::gazebo::v7::GzSceneManagerPrivate::~GzSceneManagerPrivate() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:39)
ignition::gazebo::v7::GzSceneManagerPrivate::~GzSceneManagerPrivate() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:39)
std::__1::default_delete<ignition::gazebo::v7::GzSceneManagerPrivate>::operator()(ignition::gazebo::v7::GzSceneManagerPrivate*) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1428)
std::__1::unique_ptr<ignition::gazebo::v7::GzSceneManagerPrivate, std::__1::default_delete<ignition::gazebo::v7::GzSceneManagerPrivate> >::reset(ignition::gazebo::v7::GzSceneManagerPrivate*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1689)
std::__1::unique_ptr<ignition::gazebo::v7::GzSceneManagerPrivate, std::__1::default_delete<ignition::gazebo::v7::GzSceneManagerPrivate> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
std::__1::unique_ptr<ignition::gazebo::v7::GzSceneManagerPrivate, std::__1::default_delete<ignition::gazebo::v7::GzSceneManagerPrivate> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
ignition::gazebo::v7::GzSceneManager::~GzSceneManager() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:64)
ignition::gazebo::v7::GzSceneManager::~GzSceneManager() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:64)
ignition::gazebo::v7::GzSceneManager::~GzSceneManager() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:64)
std::__1::__function::__value_func<void (void*)>::operator()(void*&&) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/functional:1885)
std::__1::function<void (void*)>::operator()(void*) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/functional:2560)
ignition::plugin::PluginWithDlHandle::~PluginWithDlHandle() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-plugin/core/src/Plugin.cc:64)
std::__1::__shared_count::__release_shared() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:2506)
std::__1::__shared_weak_count::__release_shared() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:2548)
std::__1::shared_ptr<ignition::gui::Plugin>::~shared_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:3238)
std::__1::shared_ptr<ignition::gui::Plugin>::~shared_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:3236)
ignition::gui::Application::RemovePlugin(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gui/src/Application.cc:237)
ignition::gui::Application::~Application() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gui/src/Application.cc:169)
ignition::gui::Application::~Application() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gui/src/Application.cc:158)
ignition::gui::Application::~Application() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gui/src/Application.cc:158)
std::__1::default_delete<ignition::gui::Application>::operator()(ignition::gui::Application*) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1428)
std::__1::unique_ptr<ignition::gui::Application, std::__1::default_delete<ignition::gui::Application> >::reset(ignition::gui::Application*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1689)
std::__1::unique_ptr<ignition::gui::Application, std::__1::default_delete<ignition::gui::Application> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
std::__1::unique_ptr<ignition::gui::Application, std::__1::default_delete<ignition::gui::Application> >::~unique_ptr() (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1643)
ignition::gazebo::v7::gui::runGui(int&, char**, char const*, char const*) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/Gui.cc:310)
::runGui(const char *, const char *) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/ign.cc:374)
ffi_call_unix64 (@ffi_call_unix64:25)
ffi_call_int (@ffi_call_int:182)
nogvl_ffi_call (@nogvl_ffi_call:11)
rb_nogvl (@rb_nogvl:56)
function_call (@function_call:377)
vm_call_cfunc_with_frame (@vm_call_cfunc_with_frame:88)
vm_sendish (@vm_sendish:317)
vm_exec_core (@vm_exec_core:2113)
rb_vm_exec (@rb_vm_exec:477)
rb_ec_exec_node (@rb_ec_exec_node:78)
ruby_run_node (@ruby_run_node:30)
main (@main:29)
start (@start:4)

Update thread:

The EXC_BAD_ACCESS is seen in GzSceneManager::Update after the plugin has been removed in the main thread.

std::__1::unique_ptr<ignition::gazebo::RenderUtilPrivate, std::__1::default_delete<ignition::gazebo::RenderUtilPrivate> >::operator bool() const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/memory:1674)
ignition::gazebo::v7::RenderUtil::UpdateECM(ignition::gazebo::v7::UpdateInfo const&, ignition::gazebo::v7::EntityComponentManager&) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/rendering/RenderUtil.cc:552)
ignition::gazebo::v7::GzSceneManager::Update(ignition::gazebo::v7::UpdateInfo const&, ignition::gazebo::v7::EntityComponentManager&) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/plugins/scene_manager/GzSceneManager.cc:82)
ignition::gazebo::v7::GuiRunner::Implementation::UpdatePlugins() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/GuiRunner.cc:231)
ignition::gazebo::v7::GuiRunner::Implementation::ProcessState(ignition::msgs::SerializedStepMap const&) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-gazebo/src/gui/GuiRunner.cc:220)
std::__1::__function::__value_func<void (ignition::msgs::SerializedStepMap const&, ignition::transport::v12::MessageInfo const&)>::operator()(ignition::msgs::SerializedStepMap const&, ignition::transport::v12::MessageInfo const&) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/functional:1885)
std::__1::function<void (ignition::msgs::SerializedStepMap const&, ignition::transport::v12::MessageInfo const&)>::operator()(ignition::msgs::SerializedStepMap const&, ignition::transport::v12::MessageInfo const&) const (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/functional:2560)
ignition::transport::v12::SubscriptionHandler<ignition::msgs::SerializedStepMap>::RunLocalCallback(google::protobuf::Message const&, ignition::transport::v12::MessageInfo const&) (/usr/local/include/ignition/transport12/ignition/transport/SubscriptionHandler.hh:220)
ignition::transport::v12::NodeShared::TriggerCallbacks(ignition::transport::v12::MessageInfo const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, ignition::transport::v12::NodeShared::HandlerInfo const&) (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-transport/src/NodeShared.cc:627)
ignition::transport::v12::NodeShared::RecvMsgUpdate() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-transport/src/NodeShared.cc:520)
ignition::transport::v12::NodeShared::RunReceptionTask() (/Users/rhys/Code/osrf/ign_fortress_ws/src/ign-transport/src/NodeShared.cc:364)
decltype(*(std::__1::forward<ignition::transport::v12::NodeShared*>(fp0)).*fp()) std::__1::__invoke<void (ignition::transport::v12::NodeShared::*)(), ignition::transport::v12::NodeShared*, void>(void (ignition::transport::v12::NodeShared::*&&)(), ignition::transport::v12::NodeShared*&&) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/type_traits:3635)
void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (ignition::transport::v12::NodeShared::*)(), ignition::transport::v12::NodeShared*, 2ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (ignition::transport::v12::NodeShared::*)(), ignition::transport::v12::NodeShared*>&, std::__1::__tuple_indices<2ul>) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/thread:286)
void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, void (ignition::transport::v12::NodeShared::*)(), ignition::transport::v12::NodeShared*> >(void*) (/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.0.sdk/usr/include/c++/v1/thread:297)
_pthread_start (@_pthread_start:54)
thread_start (@thread_start:8)

[chapulina]

Thanks for the ticket.

We made some updates to GUI threading on #1095, which are being ported to main on #1208. I think that may fix this issue.

[srmainwaring]

We made some updates to GUI threading on #1095, which are being ported to main on #1208. I think that may fix this issue.

Thanks, I rebased a copy of my Metal ign-gazebo repo on #1208 and cherry-picked gazebosim/gz-gui#306 into ign-gui to get the event changes. This seems to have addressed the issue in GuiRunner::UpdatePlugins() but I am still seeing a segfault in the ECM on exit (I suspect there was more than one issue and which one got hit first was largely random).

I've added the Metal changes for ign-gui to main (because they break ABI), but see that there are quite a few changes in ign-gui6 yet to be forward ported. I'm not sure whether some of these additions will solve the remaining issues?

Can you please advise which branches of ign-gui and ign-rendering I should target for the Metal changes? I might hold off investigating this issue until the pending changes to those branches have been merged and I've rebased all my changes on them.

[chapulina]

Can you please advise which branches of ign-gui and ign-rendering I should target for the Metal changes?

If you're breaking ABI on installed headers, then you'll need to target Ignition Garden.

We're currently in a transition phase for what will become Garden. Once this ticket is completed you should have a sturdy base to build on top of: gazebo-tooling/release-tools#545.

Until then... Maybe the safest approach is developing against Fortress and rebasing once Garden's rendering dependencies have been sorted out.

[chapulina]

We're currently in a transition phase for what will become Garden.

FYI, main should already be Garden

[srmainwaring]

Thanks @chapulina. By way of update on running ign-gazebo on macOS - my experimental branches seem to be very stable since rebasing on main, with no segfaults while running and only the ECM cleanup issue on exit. I think the issue has something to do with the the cleanup of the vector containing views when the ECM dtor is called - some of the pointers seem to already be invalidated which seems odd. It'll take more detailed investigation to pin the cause down but in the meanwhile it's not a show stopper for everyday use so it may be better to get the basic Metal support submitted first.

I will put PRs together for all the changes I'm using in ign-gui and ign-gazebo once ign-rendering6 is merged into main (as the downstream libraries need code from both branches). They should provide a working macOS environment with a few smaller issues to mop up as follow ups.

[chapulina]

ECM cleanup issue on exit. I think the issue has something to do with the the cleanup of the vector containing views when the ECM dtor is called - some of the pointers seem to already be invalidated which seems odd.

Not sure if it's related, but some race conditions on the views have been fixed here: #1001

[adlarkin]

ECM cleanup issue on exit. I think the issue has something to do with the the cleanup of the vector containing views when the ECM dtor is called - some of the pointers seem to already be invalidated which seems odd.

Not sure if it's related, but some race conditions on the views have been fixed here: #1001

There's another issue that I think points out the crash you're referring to, @srmainwaring: #1158

Have you by chance had any more time to look into this? If you have any more insights, feel free to comment on #1158, and we can continue the discussion there.

[srmainwaring]

Have you by chance had any more time to look into this?

@adlarkin, sorry no - it's currently in my list of 'inconvenient but not blocking' things to do with Ignition running on macOS. I plan to come back it once I've got the macOS GUI and Gazebo PR's knocked into shape and accepted.

Update

There is a workaround that prevents the segfault, on macOS at least, which I've posted as a comment in #1317. I'm happy to use this until an alternative is found, so will close this in lieu of #1158.