GCEventsListener and GCSamplerNetCore causing a hard app crash

[indy-singh]

Workaround

Setting env var in process NEW_RELIC_GC_SAMPLER_V2_ENABLED to 1 prevents NR from crashing the app.

Description
We recently converted our migrations runner from 472 to net8 and now that NR can see it (we only install NR with netcore support). It causes the migrations runner to hard crash with this stack trace:-

[exec] Fatal error. Internal CLR error. (0x80131506)
[exec] Repeat 2 times:
[exec] --------------------------------
[exec]    at System.Diagnostics.Tracing.EventPipeInternal.Enable(Char*, System.Diagnostics.Tracing.EventPipeSerializationFormat, UInt32, EventPipeProviderConfigurationNative*, UInt32)
[exec] --------------------------------
[exec]    at System.Diagnostics.Tracing.EventPipeInternal.Enable(System.String, System.Diagnostics.Tracing.EventPipeSerializationFormat, UInt32, System.Diagnostics.Tracing.EventPipeProviderConfiguration[])
[exec]    at System.Diagnostics.Tracing.EventPipeEventDispatcher.CommitDispatchConfiguration()
[exec]    at System.Diagnostics.Tracing.EventPipeEventDispatcher.SendCommand(System.Diagnostics.Tracing.EventListener, System.Diagnostics.Tracing.EventCommand, Boolean, System.Diagnostics.Tracing.EventLevel, System.Diagnostics.Tracing.EventKeywords)
[exec]    at NewRelic.Agent.Core.Samplers.GCEventsListener.StartListening()
[exec]    at NewRelic.Agent.Core.Samplers.GCSamplerNetCore.Start()
[exec]    at NewRelic.Agent.Core.Utilities.EventBus`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Publish(System.__Canon)
[exec]    at NewRelic.Agent.Core.Configuration.ConfigurationService.OnServerConfigurationUpdated(NewRelic.Agent.Core.Events.ServerConfigurationUpdatedEvent)
[exec]    at NewRelic.Agent.Core.Utilities.EventBus`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].Publish(System.__Canon)
[exec]    at NewRelic.Agent.Core.DataTransport.ConnectionHandler.Connect()
[exec]    at NewRelic.Agent.Core.DataTransport.ConnectionManager.Connect()
[exec]    at NewRelic.Agent.Core.DataTransport.ConnectionManager.LockAndConnect()
[exec]    at NewRelic.Agent.Core.Time.Scheduler+<>c__DisplayClass4_0.<ExecuteOnce>b__0()
[exec]    at NewRelic.Agent.Core.Utilities.ActionExtensions.CatchAndLog(System.Action)
[exec]    at NewRelic.Agent.Core.Time.Scheduler+<>c__DisplayClass8_0.<PrivateCreateExecuteOnceTimer>b__0(System.Object)
[exec]    at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
[exec]    at System.Threading.TimerQueueTimer.Fire(Boolean)
[exec]    at System.Threading.TimerQueue.FireNextTimers()
[exec]    at System.Threading.ThreadPoolWorkQueue.Dispatch()
[exec]    at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()

Expected Behaviour
NR should not cause the app to hard crash

Troubleshooting or NR Diag results
Enabling logging and setting it to FINEST seemed to make the problem go away (race condition?). So can't provide a log for the failure scenario. However, we do have the crash dumps that Windows takes automatically (I'll upload those shortly) when app hard crashes.

Steps to Reproduce
Pretty hard for you guys, as our migrations runner is built in house. But effectively it is a console app, that connects to PG a lot, and then runs checks to see which migrations need to be run, and then runs the delta.

Your Environment
Windows Server 2022
dotnet8
NewRelicDotNetAgent_10.41.0_x64.msi (deployed using DSC: ADDLOCAL=NETCoreSupport)
newrelic-infra-amd64.1.63.1.msi" (deployed using DSC)

[indy-singh]

Memory dumps have been added to support case Case #00264717 under account 4721360.

Cheers,
Indy

[nr-ahemsath]

Hi @indy-singh! Just letting you know we're not ignoring this. However, since you have a workaround, we're assuming you're not blocked in your use of New Relic for this service. Let us know if that's not correct.