Skip to content

[Bug]: Jest incorrectly writes new snapshots in CI #12288

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
billyjanitsch opened this issue Feb 1, 2022 · 3 comments · Fixed by #12378
Closed

[Bug]: Jest incorrectly writes new snapshots in CI #12288

billyjanitsch opened this issue Feb 1, 2022 · 3 comments · Fixed by #12378

Comments

@billyjanitsch
Copy link

Version

27.4.7

Steps to reproduce

  1. Clone https://github.com/billyjanitsch/jest-ci-snapshot-bug
  2. Run npx jest --ci. Note that the test fails, refusing to write the new snapshot.
  3. Run CI=1 npx jest. Note that the test passes, and the snapshot is written. Also note that terminal colors are disabled (CI mode).
  4. Run npx jest. Note that terminal colors are enabled (not CI mode).

Expected behavior

Step 3 should behave the same as step 2: Jest should refuse to write the snapshot and the test should fail.

Actual behavior

In step 3, the snapshot is written and the test passes.

Additional context

The issue was introduced in #9924 (as noted in #9924 (comment)), so it has been present since Jest 27.0.0. The problem is here:

https://github.com/facebook/jest/blob/161452447a8cbc071447ba92a03efc63da1dd0b7/packages/jest-config/src/normalize.ts#L1123-L1128

jest-config pulls the default value directly from argv.ci when deciding on the value of updateSnapshot.

Note that the problem isn't that Jest is failing to pick up on the CI env variable at all, because the difference between steps 3 and 4 shows that Jest is recognizing the presence of the variable and altering its behavior accordingly. It's specific to snapshot updating.

Environment

System:
    OS: macOS 10.15.7
    CPU: (16) x64 Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  Binaries:
    Node: 16.13.0 - ~/.local/share/nvm/v16.13.0/bin/node
    Yarn: 1.22.11 - /usr/local/bin/yarn
    npm: 8.3.0 - /usr/local/bin/npm
  npmPackages:
    jest: ^27.4.7 => 27.4.7
@cola119
Copy link
Contributor

cola119 commented Feb 12, 2022

I'm working on this.

@SimenB
Copy link
Member

SimenB commented Feb 15, 2022

@github-actions
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants