Skip to content

[Android] Run CoreCLR library tests on Android #114148

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 38 commits into
base: main
Choose a base branch
from

Conversation

kotlarmilos
Copy link
Member

@kotlarmilos kotlarmilos commented Apr 2, 2025

Description

This PR enables running CoreCLR library tests on the Android emulators and devices and adds a script for local testing on Windows.

Changes

  • Disabled library tests that require assemblies on disk
  • Disabled library tests that are failing on CoreCLR Android
  • Re-enabled passing tests on Android
  • Added a script for local testing on Windows

Disabled tests fall into two categories.

Globalization or other mobile‐platform issues

These tests have been disabled because they depend on invariant‐culture behaviors or platform APIs that aren’t fully supported on mobile targets:

System.Runtime\tests\System.Globalization.Tests\Invariant\Invariant.Tests.csproj
System.Net.Http\tests\FunctionalTests\System.Net.Http.Functional.Tests.csproj
System.Diagnostics.DiagnosticSource\tests\TestWithConfigSwitches\System.Diagnostics.DiagnosticSource.Switches.Tests.csproj
System.Reflection.MetadataLoadContext\tests\System.Reflection.MetadataLoadContext.Tests.csproj
System.Runtime\tests\System.Resources.ResourceManager.Tests\System.Resources.ResourceManager.Tests.csproj
System.Runtime\tests\System.Runtime.Tests\InvariantTimezone\System.Runtime.InvariantTimezone.Tests.csproj
System.Runtime.Loader\tests\System.Runtime.Loader.Tests.csproj
System.Runtime\tests\System.Text.Encoding.Tests\System.Text.Encoding.Tests.csproj
System.Runtime\tests\System.IO.Tests\System.IO.Tests.csproj
Microsoft.Bcl.Cryptography\tests\Microsoft.Bcl.Cryptography.Tests.csproj

CoreCLR Android-specific failures

The following disabled tests are specific to CoreCLR Android:

Microsoft.VisualBasic.Tests.ErrObjectTests.Clear
MonoTests.System.Configuration.mapped_ExeConfiguration_null
MonoTests.System.Configuration.OpenExeConfiguration1_UserLevel_None
System.Threading.Tests.MutexTests - add a process-wide implementation of named mutex to CoreCLR on Mono to better compatibility with Mono
System.IO.Pipelines.Tests.CompleteWithLargeWriteThrows

Closes #50871 #50874 #50923 #49937

@kotlarmilos
Copy link
Member Author

/azp run runtime-extra-platforms

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos
Copy link
Member Author

kotlarmilos commented Apr 28, 2025

With the number of disabled tests exclusively for coreclr, it feels like Android on one runtime flavor is less capable than the other. Ideally, we shouldn't need to disable any test which wasn't disabled on mono; i.e. just the $(TargetOS) == Android && $(RuntimeFlavor) == Mono to $(TargetFlavor) == Android type of updates. Can we work those differences out?

Some of the tests disabled in tests.proj are re-enabled in this PR because they are now passing in CI and updating conditions from $(TargetOS) == Android && $(RuntimeFlavor) == Mono to $(TargetFlavor) == Android wouldn't be sufficient. The changes made in this PR are necessary for those tests to run correctly on both Mono and CoreCLR. Reverting to the previous test-specific annotations would reduce the number of passing tests.

I have listed the failures that are specific to CoreCLR on Android. The remaining failures are due to a globalization issue or the Assembly.Location limitation in singlefilehost scenarios.

The number of test projects sent to helix on mono Android arm64 has increased from 217 to 243.

@kotlarmilos
Copy link
Member Author

/azp run runtime-extra-platforms

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-android,runtime-androidemulator

Copy link

Azure Pipelines successfully started running 2 pipeline(s).

@kotlarmilos
Copy link
Member Author

/azp run runtime-extra-platforms

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Microsoft.Extensions.DependencyInjection.Tests Fails on Android
6 participants