Skip to content
This repository was archived by the owner on May 1, 2024. It is now read-only.

Add scripts to run device tests #13641

Merged
merged 12 commits into from
Feb 10, 2021
Merged

Add scripts to run device tests #13641

merged 12 commits into from
Feb 10, 2021

Conversation

mattleibow
Copy link
Contributor

@mattleibow mattleibow commented Feb 3, 2021

Description of Change

This PR adds the bits required to run the device tests - either from the terminal or on CI.

I am splitting each platform into separate cake files so that the arguments can be focused and easy to use. For example, the simplest case to run tests would be:

dotnet cake eng/devices/<platform>.cake --project=$PWD/xxx/xxx.csproj

For example:

dotnet cake eng/devices/ios.cake \
  --project=$PWD/src/Platform.Handlers/tests/Xamarin.Platform.Handlers.DeviceTests.iOS/Xamarin.Platform.Handlers.DeviceTests.iOS.csproj

dotnet cake eng/devices/android.cake \
  --project=$PWD/src/Essentials/test/DeviceTests.Android/Xamarin.Essentials.DeviceTests.Android.csproj 

There is also a way just to run an output app file without building the project, then use the --app argument:

dotnet cake eng/devices/<platform>.cake --project=$PWD/xxx/xxx.csproj

For example:

dotnet cake eng/devices/ios.cake \
  --app=$PWDsrc/Platform.Handlers/tests/Xamarin.Platform.Handlers.DeviceTests.iOS/bin/iPhoneSimulator/Release/XamarinEssentialsDeviceTestsiOS.app

There is either a way to do everything and build and run in one go, or you can decide:

  • --target=test (default) - build and test
  • --target=build - just build and not run
  • --target=test --exclusive - just test and not run

In addition to what to do, you can specify more accurate paths:

  • --project=<path-to-csproj> - the path to the platform .csproj file (required)
  • --binlog=<path-to-binlog> - the path to store the binlog file of the build (default: xxx.csproj.binlog)
  • --app=<pth-to-apk-or-app> - the path to the .app folder o the .apk file (default: the first app/apk in the bin directory)
  • --results=<path-to-results-dir> - the path to a directory that will hold all the test results and device logs (default: xxx.app-results)
  • --device=<device-combo> - the target ID of the device used by xharness in the form
    <platform>-<device|simulator|emulator>[-<32|64>][_<ios-version|android-api>]
    eg: ios-simulator-64_13.4,android-emulator-32_30,[...]
  • --package=<android-package-name> - the Android app package name found in the manifest
  • --instrumentation=<android-instrumentation-name> - the Android instrumentation name

Issues Resolved

API Changes

None

Platforms Affected

None

Behavioral/Visual Changes

None

Before/After Screenshots

Not applicable

Testing Procedure

The cake files can just be run using either the default args or any set of valid, custom args. This should not require any other work - except maybe to install the emulators using the SDK manager. They are created, started and deleted after each run.

PR Checklist

  • Targets the correct branch
  • Tests are passing (or failures are unrelated)

@mattleibow
Copy link
Contributor Author

Tis needs to be added to the CI yaml, but that should probably be done in a separate PR.

@mattleibow mattleibow marked this pull request as ready for review February 4, 2021 00:22
@jsuarezruiz jsuarezruiz added the t/housekeeping ♻︎ Internal only changes, won't be included in release notes label Feb 4, 2021
@rmarinho rmarinho requested review from PureWeen and rmarinho February 4, 2021 11:04
@rmarinho
Copy link
Member

rmarinho commented Feb 4, 2021

I wonder if the build of the project that we have in the other cake script should be used..

@mattleibow
Copy link
Contributor Author

mattleibow commented Feb 6, 2021

I wonder if the build of the project that we have in the other cake script should be used..

That can be done too. You can pass in the app package instead of the project and then add the --exclusive arg. This will skip the build and run the tests directly. We could potentially have a bunch of jobs that pop out different variants of some app, and then just run all the tests on each app.

@PureWeen PureWeen requested a review from rmarinho February 10, 2021 21:20
@mattleibow mattleibow merged commit 678aff8 into main-handler Feb 10, 2021
@mattleibow mattleibow deleted the dev/device-tests branch February 10, 2021 23:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
t/housekeeping ♻︎ Internal only changes, won't be included in release notes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants