Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: Running with --coverage modifies binding names and results in test failures #15554

Open
ryanwilsonperkin opened this issue Mar 28, 2025 · 1 comment

Comments

@ryanwilsonperkin
Copy link
Contributor

Version

29.7.0

Steps to reproduce

Minimal reproduction available:

  1. npx github:ryanwilsonperkin/jest-coverage-typeerror passes
  2. npx github:ryanwilsonperkin/jest-coverage-typeerror --coverage fails

Expected behavior

I expect that a test that passes without --coverage enabled will still pass when it is enabled

Actual behavior

The test which asserts on the message in a thrown TypeError will fail, because the name of the original object under test has been modified from "o" to "((cov_23dwi37auj(...).s[0]++) , o)"

Additional context

No response

Environment

System:
    OS: macOS 15.3.1
    CPU: (11) arm64 Apple M3 Pro
  Binaries:
    Node: 23.5.0 - /opt/homebrew/bin/node
    Yarn: 1.22.22 - /opt/homebrew/bin/yarn
    npm: 10.9.2 - /opt/homebrew/bin/npm
  npmPackages:
    jest: ^29.7.0 => 29.7.0
@blended-bram
Copy link

Doesn't this mean your test is reliant on implementation details of the JS engine, rather than the actual behavior you want to test? Meaning, does this problem generalize to testing ones own code?

In this case the source transform which normally preserves as many semantics as possible, fails to retain the semantic that a TypeError thrown does not reflect on the right name of the variable having a field-read failure.
That is a detail of the JS engine revealing variable names for this kind of TypeError; and that is the detail the test fails on.
The test could be constructed to not verify this property, while still verifying all other semantics of the function itself.
Your test could assert .toThrow(TypeError) in order to verify that the type of error is correct and/or use the regex matcher variation to assert that it has the various pieces in the error text such as `"Cannot destructure property 'value' of".

The meaning of your test then is to verify not the behavior of the function under test accessProperty, but instead you are running into the semantics of the JS runtime.

The function called is throwing all the same, with a very similar message in fact; it is just slightly different.
But is that difference meaningful?

If the problem generalizes then it should be possible to find a similar problem where the failure derives from actual logic in the functions under test.

To make the case that the difference you observed is meaningful, would mean that you have written code that will decide differently by this detail. Meaning, you programmed something that makes a decision based on the name of the variable for which the field access is throwing for. If you write code that actually logically depends on the variable name reflecting the failure of transformed code; then that should be relevant context. All the same, this is suggesting some strange programming practice, even by JS standards.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants