.NET 9.0.1 Breaking Change in macOS native library resolution

[yaakov-h]

Description

.NET runtime 9.0.1 seems to have introduced a breaking change with regards to how dyld on macOS resolves relative paths to system frameworks.

I suspect this will be related to the change in pipelines from macos-12 to macos-13:

        # OSX Public Build Pool (we don't have on-prem OSX BuildPool).
        ${{ if and(in(parameters.osGroup, 'osx', 'maccatalyst', 'ios', 'iossimulator', 'tvos', 'tvossimulator'), eq(variables['System.TeamProject'], 'public')) }}:
-          vmImage: 'macos-12'
+          vmImage: 'macos-13'

        # OSX Internal Pool
        ${{ if and(in(parameters.osGroup, 'osx', 'maccatalyst', 'ios', 'iossimulator', 'tvos', 'tvossimulator'), ne(variables['System.TeamProject'], 'public')) }}:
          name: "Azure Pipelines"
-          vmImage: 'macOS-12'
+          vmImage: 'macOS-13'
          os: macOS

        # Official Build Windows Pool

(I don't know enough about the .NET build engineering side so I could be wrong here, but that is definitely sus.)

Is there a way to restore the old behaviour for the .NET 9 lifecycle?

If not, should this at least be documented somewhere? I didn't see this in any of the release notes.

See SteamRE/SteamKit#1501 and Homebrew/homebrew-core#206331 (comment) for details.

Reproduction Steps

Attempt to DllImport/LibraryImport a system framework using a relative path on macOS e.g. CoreFoundation.framework/CoreFoundation

Expected behavior

The P/Invoke function is loaded and executed.

Actual behavior

System.DllNotFoundException : Unable to load shared library 'CoreFoundation.framework/CoreFoundation' or one of its dependencies. In order to help diagnose loading problems, consider setting the DYLD_PRINT_LIBRARIES environment variable: 
dlopen(/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation.dylib, 0x0001): tried: '/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation.dylib' (no such file), '/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation.dylib' (no such file)
dlopen(/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation.dylib, 0x0001): tried: '/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation.dylib' (no such file), '/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation.dylib' (no such file)
dlopen(CoreFoundation.framework/CoreFoundation.dylib, 0x0001): tried: 'CoreFoundation.framework/CoreFoundation.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OSCoreFoundation.framework/CoreFoundation.dylib' (no such file), '/usr/lib/CoreFoundation.framework/CoreFoundation.dylib' (no such file, not in dyld cache), 'CoreFoundation.framework/CoreFoundation.dylib' (no such file)
dlopen(/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation, 0x0001): tried: '/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation' (no such file), '/Users/runner/.dotnet/shared/Microsoft.NETCore.App/9.0.1/CoreFoundation.framework/CoreFoundation' (no such file)
dlopen(/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation, 0x0001): tried: '/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation' (no such file), '/Users/runner/work/SteamKit/SteamKit/SteamKit2/Tests/bin/Debug/net8.0/CoreFoundation.framework/CoreFoundation' (no such file)
dlopen(CoreFoundation.framework/CoreFoundation, 0x0001): tried: 'CoreFoundation.framework/CoreFoundation' (no such file), '/System/Volumes/Preboot/Cryptexes/OSCoreFoundation.framework/CoreFoundation' (no such file), '/usr/lib/CoreFoundation.framework/CoreFoundation' (no such file, not in dyld cache), 'CoreFoundation.framework/CoreFoundation' (no such file)

Regression?

This worked in .NET 8 and seems to also work in .NET 9.0.0. It fails in .NET 9.0.1.

Known Workarounds

Workaround 1: Launch with the environment variable DYLD_FRAMEWORK_PATH=/System/Library/Frameworks preset.

Workaround 2: Recompile and redeploy binaries with DllImport/LibraryImport set to an absolute path to the framework binaries, rather than a relative path.

Configuration

macos-14 and macos-latest on GitHub Actions, as well as my MacBook Pro (M2 Pro) with Sequoia 15.3 (24D60) arm64

.NET 9.0.1

Other information

cc @dotnet/compat

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

[AaronRobinsonMSFT]

@elinor-fung Thoughts on what might have changed here?

[yaakov-h]

FWIW I'm also seeing this now on .NET 8.0.12.

If I run the failing unit test with DOTNET_ROLL_FORWARD=disable it passes, with DOTNET_ROLL_FORWARD=LatestPatch it fails.

I suspect that with <RollForward>LatestMajor</RollForward> in the test .csproj it was rolling forward to 9.0 so I'm not sure which patch of .NET 8 introduced the same bug as I tried to set several but they would have all been skipped due to rollforward behaviour.

Though given that the same vmImage update occurred between .NET 8.0.11 and .NET 8.0.12, I would put my money on that...

[agocke]

I don’t think we changed anything in the code, so I agree that if something changed the OS image update looks notable. But I have no idea what could have changed in 13 to cause this type of code change.

[ejhg]

I am seeing this issue in sdk 8.0.405 and 8.0.406. (Issue is absent in 8.0.404 and prior.)

[DennisDyallo]

We're seeing this issue in the Yubico.NET.SDK which loads some native libraries in order to do some native work (connecting to USB-devices, CFString, etc.)

We came across the error when upgrading our integration test and example projects to .net8 and .net9 and running on Mac (Sequoia 15.2, 15.3.1)

While these workarounds work fine in our own integration testing, it wont work well downstream with the end users of our SDK/3rd party client apps.

Would like to see a confirmation from MSFT on the issue, the intended fix and forward guidance.