Skip to content

Integration tests fail after upgrading to XUnit 2.9.0 #2684

Closed
@tippmar-nr

Description

@tippmar-nr

Description
Several integration tests fail consistently after upgrading to XUnit 2.9.0, with errors similar to the following:

  Error Message:
   System.InvalidOperationException : There is no currently active test.
  Stack Trace:
     at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.WaitForLogLines(String regularExpression, Nullable`1 timeoutOrZero, Int32 minimumCount) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 169
   at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.GetReportingAppLink(Nullable`1 timeoutOrZero) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 118
   at NewRelic.Agent.IntegrationTestHelpers.AgentLogBase.GetAccountId(Nullable`1 timeoutOrZero) in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTestHelpers\AgentLogBase.cs:line 93
   at NewRelic.Agent.IntegrationTests.WCF.Client.WCFClientTestBase.DistributedTracing_Metrics() in D:\a\newrelic-dotnet-agent\newrelic-dotnet-agent\tests\Agent\IntegrationTests\IntegrationTests\WCF\WCFClientTests.cs:line 130

The XUnit release notes for v2.9.0 indicate that they fixed a bug related to re-using instances of TestOutputHelper between tests. Prior to the fix, reusing an instance would (incorrectly) mix log output between tests; after the fix, the InvalidOperationException shown above is thrown.

The errors occur primarily in the DistributedTracing and WCF integration test namespaces, indicating that we need to review the implementation of those tests and fix any reuse of TestOutputHelper instances between tests. This comment on the XUnit issue may be helpful in our analysis.

Expected Behavior
All integration tests should pass after upgrading to XUnit v2.9.0

For Maintainers Only or Hero Triaging this bug
Suggested Priority (P1,P2,P3,P4,P5):
P3
Suggested T-Shirt size (S, M, L, XL, Unknown):
M

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions