Try to re-enable APICompat in aspnetcore

[ViktorHofer]

Tracking issue: #45004

AspNetCore builds with desktop msbuild

[ericstj]

/azp run sdk-unified-build

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[ericstj]

This should fix it, assuming we use live SDK (or task). If not, we need to wait for the SDK used by unified build to be updated.

[ViktorHofer]

We don't use live. AspNetCore builds before sdk. We need a toolset update.

[ericstj]

How do we update the toolset used by the unified build? Is that just the toolset used in SDK, some other place specific to UB, or do we need to update every repo?

[ViktorHofer]

638f4f1 shows a re-bootstrap change. Such a PR will get opened through a special pipeline: https://github.com/dotnet/source-build/blob/main/Documentation/VMR-re-bootstrapping.md#steps-to-re-bootstrap

I only got another APICompat change merged earlier today and the dotnet-source-build pipeline that is used for re-bootstrapping runs once per day. I think we should wait until Monday with re-boostrapping.

[Copilot]

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)

• src/SourceBuild/content/repo-projects/aspnetcore.proj: Language not supported

[ViktorHofer]

Looks like the APICompat issue or another variant of it still exists in the product. @ericstj would you mind taking another look?

[ericstj]

Is the build of this SDK "special" in any way? Where can I get the binaries?

    D:\a\_work\1\vmr\.dotnet\sdk\10.0.100-preview.2.25112.53\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ApiCompat.ValidatePackage.targets(39,5): error MSB4018: The "Microsoft.DotNet.ApiCompat.Task.ValidatePackageTask" task failed unexpectedly. [D:\a\_work\1\vmr\src\aspnetcore\src\DataProtection\Cryptography.Internal\src\Microsoft.AspNetCore.Cryptography.Internal.csproj]
    D:\a\_work\1\vmr\.dotnet\sdk\10.0.100-preview.2.25112.53\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.ApiCompat.ValidatePackage.targets(39,5): error MSB4018: System.MissingMethodException: Method not found: 'Void Microsoft.CodeAnalysis.CSharp.CSharpCompilationOptions..ctor(Microsoft.CodeAnalysis.OutputKind, Boolean, System.String, System.String, System.String, System.Collections.Generic.IEnumerable`1<System.String>, Microsoft.CodeAnalysis.OptimizationLevel, Boolean, Boolean, System.String, System.String, System.Collections.Immutable.ImmutableArray`1<Byte>, System.Nullable`1<Boolean>, Microsoft.CodeAnalysis.Platform, Microsoft.CodeAnalysis.ReportDiagnostic, Int32, System.Collections.Generic.IEnumerable`1<System.Collections.Generic.KeyValuePair`2<System.String,Microsoft.CodeAnalysis.ReportDiagnostic>>, Boolean, Boolean, Microsoft.CodeAnalysis.XmlReferenceResolver, Microsoft.CodeAnalysis.SourceReferenceResolver, Microsoft.CodeAnalysis.MetadataReferenceResolver, Microsoft.CodeAnalysis.AssemblyIdentityComparer, Microsoft.CodeAnalysis.StrongNameProvider, Boolean, Microsoft.CodeAnalysis.MetadataImportOptions, Microsoft.CodeAnalysis.NullableContextOptions)'. [D:\a\_work\1\vmr\src\aspnetcore\src\DataProtection\Cryptography.Internal\src\Microsoft.AspNetCore.Cryptography.Internal.csproj]

Update -- found in logs -- https://ci.dot.net/public/Sdk/10.0.100-preview.2.25112.53/dotnet-sdk-10.0.100-preview.2.25112.53-win-x64.zip

[ericstj]

It could be that Roslyn is now using a newer version of one of these dependencies than is redirected by MSBuild.

[ericstj]

This is interesting --

In the same binlog there are successful runs and failures. It seems to be about 50/50

I'm also seeing this in the same log -- compiler toolset package

So two different versions of roslyn loaded depending on the project built. I wonder if this has something to do with it? Perhaps the task binds to a different roslyn version and future runs of that task fail if using a different version?

The types involved with this method signature come from Microsoft.CodeAnalysis and System.Collections.Immutable. I think with the changes from #46019 both task and Roslyn should be referencing a version of SCI <= MSBuild's bindingRedirects - so both should get the same version. The mismatch of MCS should have been solved with #22277 - but given we have a toolset package involved here it seems fishy enough that it could be related.

[ericstj]

I found what's happening here.

In this situation the SDK was built with a newer MSBuild than the VS it is running on that triggers this:

sdk/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Sdk.targets

Lines 223 to 236 in bacea4e

	<PropertyGroup Condition="'$(MSBuildRuntimeType)' == 'Full'">
	<!-- Automatically opt users into using the toolset package if they are running an MSBuild other than what this SDK was built against.
	This is to reduce 'tearing'/dependency mismatch, but as always users can override this behavior by disabling the hosted compiler flag. -->
	<BuildWithNetFrameworkHostedCompiler Condition="'$(BuildWithNetFrameworkHostedCompiler)' == ''
	and '$(_IsDisjointMSBuildVersion)' == 'true'
	and ('$(Language)' == 'C#'
	or '$(Language)' == 'VB')">true</BuildWithNetFrameworkHostedCompiler>
	</PropertyGroup>
	<PropertyGroup Condition="'$(BuildWithNetFrameworkHostedCompiler)' == 'true' and '$(OS)' == 'Windows_NT'">
	<RoslynTargetsPath>$(NuGetPackageRoot)\microsoft.net.sdk.compilers.toolset\$(NETCoreSdkVersion)</RoslynTargetsPath>
	<_NeedToDownloadMicrosoftNetSdkCompilersToolsetPackage>true</_NeedToDownloadMicrosoftNetSdkCompilersToolsetPackage>
	<_MicrosoftNetSdkCompilersToolsetPackageRootEmpty Condition="'$(NuGetPackageRoot)' == ''">true</_MicrosoftNetSdkCompilersToolsetPackageRootEmpty>
	</PropertyGroup>

But only in inner-builds since these targets are only imported in inner builds. The result is that any multi-targeted project in aspnetcore will run ApiCompat with MSBuild's roslyn (version A), where non-multi-targeted projects run with the toolset package's Roslyn (version B). Add to this that non-multi-targeted projects create zero work for ApiCompat since Baseline checking is disabled. This causes APICompat to only load Micrsoft.CodeAnalysis and not Microsoft.CodeAnalysis.CSharp from the toolset package Version B. Later on, the same MSBuild node (same appdomain) builds a multi-targeting project - so it has work for APICompat and uses version A. The APICompat assembly already bound to Microsoft.CodeAnalysis version B, but since it hasn't bound to Microsoft.CodeAnalysis.CSharp yet that triggers a resolve which will be resolved to version A - even though version B was loaded in the appdomain, nothing redirected to it. Now the PR #22277 was meant to solve for this situation by pre-loading both assemblies, but the problem is that just loading isn't really good enough if the version differs. What needs to happen is the actual load of version X needs to trigger an AssemblyResolve that gets resolved to version Y.

There are a number of ways to fix this. To workaround we can either disable the toolset compiler, or lift the same logic that was in the inner-build to outerbuild for selecting a toolset roslyn package.

To fix this in APICompat we can do one of two things:

1. Ensure we bind to a single version of all of roslyn's assemblies by either forcing a load, or using a write-once static to capture the Roslyn assemblies path.
2. Additionally to 1, create an appdomain to host APICompat assemblies if the version of roslyn specified differs from the one already set.

[ericstj]

Now all of the failures are

The analyzer assembly 'D:\a_work\1\vmr.dotnet\sdk\10.0.100-preview.2.25112.53\Sdks\Microsoft.NET.Sdk\codestyle\cs\Microsoft.CodeAnalysis.CodeStyle.dll' references version '4.14.0.0' of the compiler, which is newer than the currently running version '4.13.0.0'.

So it would seem we must go the other way and force the toolset compiler to be used in outer build. I will see about the best way to do that.