Skip to content

Commit ec8c456

Browse files
warnerwarner
committed
test: set ESM_DISABLE_CACHE to maybe fix CI-vs--r-esm
We were seeing random test failures in the zoe unit tests (specifically test-offerSafety.js) that looked like: ``` Uncaught exception in test/unitTests/test-offerSafety.js /home/runner/work/agoric-sdk/agoric-sdk/packages/weak-store/src/weakStore.js:5 import { assert, details, q } from '@agoric/assert'; ^^^^^^ SyntaxError: Cannot use import statement outside a module ✖ test/unitTests/test-offerSafety.js exited with a non-zero exit code: 1 ``` or: ``` Uncaught exception in test/unitTests/test-offerSafety.js /home/runner/work/agoric-sdk/agoric-sdk/packages/zoe/test/unitTests/setupBasicMints.js:1 import { makeIssuerKit } from '@agoric/ertp'; SyntaxError: Cannot use import statement outside a module ✖ test/unitTests/test-offerSafety.js exited with a non-zero exit code: 1 ``` under various versions of Node.js. The error suggests that `-r esm` was not active (the error site tried to import an ESM-style module, and failed because the import site was CJS not ESM), however error site itself was in a module, meaning `-r esm` was active up until that moment. This makes no sense. The AVA runner has had some amount of built-in support for ESM, in which it uses Babel to rewrite test files (but perhaps not the code being tested). If that were in effect, it might explain how `test-offerSafety.js` could be loaded, but `weakStore.js` could not. It is less likely to explain how `setupBasicMints.js` could be loaded but `makeIssuerKit` could not, unless AVA somehow believes that `setupBasicMints.js` is a different category of file than the ones pulled from other packages. In any case, I believe we're bypassing that built-in/Babel support, by using their recommended `-r esm` integration recipe (package.json has `ava.require=['esm']`). However the AVA "ESM Plan" (avajs/ava#2293) is worth reading, if only for our future move-to-native-ESM plans (#527). So my hunch here is that the `-r esm` module's cache is not safe against concurrent access, and AVA's parallel test invocation means there are multiple processes reading and writing to that cache in parallel. Zoe has more source files than most other packages, which might increase the opportunity for a cache-corruption bug to show up. This sort of bug might not show up locally because the files are already in the cache, whereas CI may not already have them populated. This patch adds `ESM_DISABLE_CACHE=true` to the environment variables used in all tests, in an attempt to avoid this hypothetical bug. Stale entries in this cache has caused us problems before, so most of us have the same setting in our local shells. Another potential workaround would be to add `--serial` to the `ava` invocation, however the Zoe test suite is large enough that we really to want the parallelism, just to make the tests finish faster. This patch also increases the Zoe test timeout to 10m, just in case. I observed a few tests taking 1m30s or 1m40s to complete, and the previous timeout was 2m, which was too close to the edge.
1 parent 45015dc commit ec8c456

File tree

2 files changed

+69
-2
lines changed

2 files changed

+69
-2
lines changed

‎.github/workflows/test-all-packages.yml

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ on:
88
branches: [master]
99
pull_request:
1010

11+
# set ESM_DISABLE_CACHE=true (will be JSON parsed)
1112
jobs:
1213
build:
1314
runs-on: ubuntu-latest
@@ -110,64 +111,124 @@ jobs:
110111
# run: yarn test
111112
- name: yarn test (acorn-eventual-send)
112113
run: cd packages/acorn-eventual-send && yarn test
114+
env:
115+
ESM_DISABLE_CACHE: true
113116
- name: yarn test (agoric-cli)
114117
run: cd packages/agoric-cli && yarn test
118+
env:
119+
ESM_DISABLE_CACHE: true
115120
- name: yarn test (assert)
116121
run: cd packages/assert && yarn test
122+
env:
123+
ESM_DISABLE_CACHE: true
117124
- name: yarn test (bundle-source)
118125
run: cd packages/bundle-source && yarn test
126+
env:
127+
ESM_DISABLE_CACHE: true
119128
- name: yarn test (captp)
120129
run: cd packages/captp && yarn test
130+
env:
131+
ESM_DISABLE_CACHE: true
121132
- name: yarn test (dapp-svelte-wallet/api)
122133
run: cd packages/dapp-svelte-wallet/api && yarn test
134+
env:
135+
ESM_DISABLE_CACHE: true
123136
- name: yarn test (deployment)
124137
run: cd packages/deployment && yarn test
138+
env:
139+
ESM_DISABLE_CACHE: true
125140
- name: yarn test (ERTP)
126141
run: cd packages/ERTP && yarn test
142+
env:
143+
ESM_DISABLE_CACHE: true
127144
- name: yarn test (eventual-send)
128145
run: cd packages/eventual-send && yarn test
146+
env:
147+
ESM_DISABLE_CACHE: true
129148
- name: yarn test (import-bundle)
130149
run: cd packages/import-bundle && yarn test
150+
env:
151+
ESM_DISABLE_CACHE: true
131152
- name: yarn test (import-manager)
132153
run: cd packages/import-manager && yarn test
154+
env:
155+
ESM_DISABLE_CACHE: true
133156
- name: yarn test (install-metering-and-ses)
134157
run: cd packages/install-metering-and-ses && yarn test
158+
env:
159+
ESM_DISABLE_CACHE: true
135160
- name: yarn test (install-ses)
136161
run: cd packages/install-ses && yarn test
162+
env:
163+
ESM_DISABLE_CACHE: true
137164
- name: yarn test (marshal)
138165
run: cd packages/marshal && yarn test
166+
env:
167+
ESM_DISABLE_CACHE: true
139168
- name: yarn test (notifier)
140169
run: cd packages/notifier && yarn test
170+
env:
171+
ESM_DISABLE_CACHE: true
141172
- name: yarn test (promise-kit)
142173
run: cd packages/promise-kit && yarn test
174+
env:
175+
ESM_DISABLE_CACHE: true
143176
- name: yarn test (registrar)
144177
run: cd packages/registrar && yarn test
178+
env:
179+
ESM_DISABLE_CACHE: true
145180
- name: yarn test (same-structure)
146181
run: cd packages/same-structure && yarn test
182+
env:
183+
ESM_DISABLE_CACHE: true
147184
- name: yarn test (sharing-service)
148185
run: cd packages/sharing-service && yarn test
186+
env:
187+
ESM_DISABLE_CACHE: true
149188
- name: yarn test (sparse-ints)
150189
run: cd packages/sparse-ints && yarn test
190+
env:
191+
ESM_DISABLE_CACHE: true
151192
- name: yarn test (spawner)
152193
run: cd packages/spawner && yarn test
194+
env:
195+
ESM_DISABLE_CACHE: true
153196
- name: yarn test (stat-logger)
154197
run: cd packages/stat-logger && yarn test
198+
env:
199+
ESM_DISABLE_CACHE: true
155200
- name: yarn test (store)
156201
run: cd packages/store && yarn test
202+
env:
203+
ESM_DISABLE_CACHE: true
157204
- name: yarn test (swing-store-lmdb)
158205
run: cd packages/swing-store-lmdb && yarn test
206+
env:
207+
ESM_DISABLE_CACHE: true
159208
- name: yarn test (swing-store-simple)
160209
run: cd packages/swing-store-simple && yarn test
210+
env:
211+
ESM_DISABLE_CACHE: true
161212
- name: yarn test (swingset-runner)
162213
run: cd packages/swingset-runner && yarn test
214+
env:
215+
ESM_DISABLE_CACHE: true
163216
- name: yarn test (tame-metering)
164217
run: cd packages/tame-metering && yarn test
218+
env:
219+
ESM_DISABLE_CACHE: true
165220
- name: yarn test (transform-eventual-send)
166221
run: cd packages/transform-eventual-send && yarn test
222+
env:
223+
ESM_DISABLE_CACHE: true
167224
- name: yarn test (transform-metering)
168225
run: cd packages/transform-metering && yarn test
226+
env:
227+
ESM_DISABLE_CACHE: true
169228
- name: yarn test (weak-store)
170229
run: cd packages/weak-store && yarn test
230+
env:
231+
ESM_DISABLE_CACHE: true
171232

172233
##############
173234
# Long-running tests are executed individually.
@@ -212,6 +273,8 @@ jobs:
212273
${{ runner.os }}-go-
213274
- name: yarn test (cosmic-swingset)
214275
run: cd packages/cosmic-swingset && yarn test
276+
env:
277+
ESM_DISABLE_CACHE: true
215278

216279
test-swingset:
217280
# BEGIN-TEST-BOILERPLATE
@@ -243,6 +306,8 @@ jobs:
243306
# END-RESTORE-BOILERPLATE
244307
- name: yarn test (SwingSet)
245308
run: cd packages/SwingSet && yarn test
309+
env:
310+
ESM_DISABLE_CACHE: true
246311

247312
test-zoe:
248313
# BEGIN-TEST-BOILERPLATE
@@ -274,3 +339,5 @@ jobs:
274339
# END-RESTORE-BOILERPLATE
275340
- name: yarn test (zoe)
276341
run: cd packages/zoe && yarn test
342+
env:
343+
ESM_DISABLE_CACHE: true

‎packages/zoe/package.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
},
99
"scripts": {
1010
"build": "yarn build-zcfBundle",
11-
"test": "ava",
11+
"test": "ava --verbose",
1212
"build-zcfBundle": "node -r esm scripts/build-zcfBundle.js",
1313
"lint-fix": "yarn lint --fix",
1414
"lint-check": "yarn lint",
@@ -63,7 +63,7 @@
6363
"ava": {
6464
"files": ["test/**/test-*.js"],
6565
"require": ["esm"],
66-
"timeout": "2m"
66+
"timeout": "10m"
6767
},
6868
"eslintConfig": {
6969
"extends": [

0 commit comments

Comments
 (0)