Skip to content

winch(aarch64): Run wast test in CI #10750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

saulecabrera
Copy link
Member

Closes #9566

This commit takes inspiration from
https://github.com/bytecodealliance/wasmtime/pull/10738/files

During the 2025-05-07 Cranelift meeting we discussed potential avenues to start running wast tests for winch-aarch64. One potential idea was to declare an allowlist of tests that should be exectued and potentially ignore everyhing else. Although that idea works, I decided to diverge a bit from it, in favor of introducing a very strict classification of the state of tests for aarch64, namely, in this commit tests are classified as:

  • Run and expected to pass
  • Run and exepected to fail in a recoverable way
  • Don't run since it's known that they produce an unrecoverable error e.g., segafault.

This approach is probably more verbose than the one discussed in the meeting, however, I think it's easier to have a global view of that status for aarch64 and potentially other backends in the future.

@alexcrichton curious to see what do you think.

Closes bytecodealliance#9566

This commit takes inspiration from
https://github.com/bytecodealliance/wasmtime/pull/10738/files

During the 2025-05-07 Cranelift meeting we discussed potential avenues
to start running wast tests for winch-aarch64. One potential idea was
to declare an allowlist of tests that should be exectued and
potentially ignore everyhing else. Although that idea works, I decided
to diverge a bit from it, in favor of introducing a very strict
classification of the state of tests for aarch64, namely, in this
commit tests are classified as:

* Run and expected to pass
* Run and exepected to fail in a recoverable way
* Don't run since it's known that they produce an unrecoverable error
e.g., segafault.

This approach is probably more verbose than the one discussed in the
meeting, however, I think it's easier to have a global view of that
status for aarch64 and potentially other backends in the future.
@saulecabrera saulecabrera requested a review from a team as a code owner May 7, 2025 22:35
@saulecabrera saulecabrera requested review from fitzgen and removed request for a team May 7, 2025 22:35
@saulecabrera saulecabrera added this pull request to the merge queue May 7, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks May 7, 2025
@github-actions github-actions bot added wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime labels May 8, 2025
Copy link

github-actions bot commented May 8, 2025

Label Messager: wasmtime:config

It looks like you are changing Wasmtime's configuration options. Make sure to
complete this check list:

  • If you added a new Config method, you wrote extensive documentation for
    it.

    Our documentation should be of the following form:

    Short, simple summary sentence.
    
    More details. These details can be multiple paragraphs. There should be
    information about not just the method, but its parameters and results as
    well.
    
    Is this method fallible? If so, when can it return an error?
    
    Can this method panic? If so, when does it panic?
    
    # Example
    
    Optional example here.
    
  • If you added a new Config method, or modified an existing one, you
    ensured that this configuration is exercised by the fuzz targets.

    For example, if you expose a new strategy for allocating the next instance
    slot inside the pooling allocator, you should ensure that at least one of our
    fuzz targets exercises that new strategy.

    Often, all that is required of you is to ensure that there is a knob for this
    configuration option in wasmtime_fuzzing::Config (or one
    of its nested structs).

    Rarely, this may require authoring a new fuzz target to specifically test this
    configuration. See our docs on fuzzing for more details.

  • If you are enabling a configuration option by default, make sure that it
    has been fuzzed for at least two weeks before turning it on by default.


To modify this label's message, edit the .github/label-messager/wasmtime-config.md file.

To add new label messages or remove existing label messages, edit the
.github/label-messager.json configuration file.

Learn more.

@saulecabrera
Copy link
Member Author

saulecabrera commented May 8, 2025

The failure in CI happens because wasmtime_test_util::wast::Compiler::should_fail gets invoked from the wasmtime_test macro, and as far as I understand, #[cfg(target_arch="...")] in macro code will resolve to the host, not to the target, which ends up returning true for should_fail, but since we're cross compiling (aarch64 -> x64), in reality the test passes and the #[should_panic] predicate is not met.

Prior to this change this was not an issue, since we were only taking x64 into account for Winch.

@alexcrichton
Copy link
Member

Bah I was afraid of this happening at some point...

Could we perhaps delay this to happening at runtime? We've got the two structures there at test-run-time so we should be able to run the same code. I'd prefer to not catch panics so it means though that all tests would have to return Result<()> and the runtime result of should_fail would result in expecting an error or not, however.

@saulecabrera
Copy link
Member Author

Yeah, I'll try that. I remember we had discussed this, I just wasn't sure if there was a strong preference regarding catching panics.

@alexcrichton
Copy link
Member

Personally I'd say I have a weak preference, if that's the best solution maintenance-wise I'd go with catching panics

`Compiler::should_fail` makes use of `#[cfg(target_arg="...")]` directives to
derive the compiler-arch-Wasm proposal support matrix, however,
`Compiler::should_fail` is also called from the `wasmtime_test` macro
which introduces conflicts with the `target_arch` resolution (within macros,
these directives resolve to the host compiler, instead of the target).

This commit emits `Compiler::should_fail` at runtime instead of at
compile time, which fixes the issue mentioned above.
@saulecabrera saulecabrera force-pushed the winch-aarch64-run-more-tests branch from d810db4 to ad0e177 Compare May 8, 2025 18:41
@saulecabrera
Copy link
Member Author

Ok, ended up going with emitting the call to Compiler::should_fail at runtime. Would you mind taking another look?

@saulecabrera saulecabrera force-pushed the winch-aarch64-run-more-tests branch from 8fe446d to 34c3676 Compare May 9, 2025 13:06
@saulecabrera saulecabrera added this pull request to the merge queue May 9, 2025
Merged via the queue into bytecodealliance:main with commit 35053d6 May 9, 2025
41 checks passed
@saulecabrera saulecabrera deleted the winch-aarch64-run-more-tests branch May 9, 2025 13:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
wasmtime:api Related to the API of the `wasmtime` crate itself wasmtime:config Issues related to the configuration of Wasmtime
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Winch support for AArch64 tough to integrate with tests/wast.rs
2 participants