Skip to content

test_runner: add global setup and teardown functionality #57438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 35 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
34b538a
test_runner: add global setup and teardown functionality
pmarchini Apr 9, 2025
71dc16d
test_runner: move global setup and teardown to harness
pmarchini Apr 9, 2025
56017c3
test_runner: implement global setup and teardown functionality in har…
pmarchini Apr 9, 2025
c3fda43
test_runner: add tests for global setup and teardown hooks in watch mode
pmarchini Apr 9, 2025
274dee0
test_runner: add documentation and option for global setup script
pmarchini Apr 9, 2025
ed7f2f8
test_runner: ensure globalSetup runs only once per test run
pmarchini Apr 9, 2025
7e4da85
test_runner: add --import with tests interop test
pmarchini Apr 9, 2025
85525d1
test_runner: modify global setup behavior based on test option
pmarchini Apr 9, 2025
c978cb7
test_runner: ensure globalTeardown is being executed after all the tests
pmarchini Apr 9, 2025
35b9128
test_runner: improve documentation for global setup and teardown func…
pmarchini Apr 9, 2025
5f4831d
test_runner: lint cpp
pmarchini Apr 9, 2025
1b4a328
test: implement running each test in a separate process
pmarchini Apr 9, 2025
d822658
test_runner: fix cjs/mjs doc
pmarchini Apr 9, 2025
bb35899
test: use fileURL for --import
pmarchini Apr 9, 2025
8fd0fb9
fixup! test_runner: add documentation and option for global setup script
pmarchini Apr 9, 2025
f3025df
test_runner: revert formatter
pmarchini Apr 9, 2025
795f734
test_runner: streamline global setup and teardown functions
pmarchini Apr 9, 2025
cf3d8e8
test_runner: move global setup watch mode test to single file
pmarchini Apr 9, 2025
e8f8860
test_runner: refactor global setup and teardown
pmarchini Apr 9, 2025
5437dd2
test_runner: update fixtures
pmarchini Apr 9, 2025
c15a54b
test: remove IBMi and AIX
pmarchini Apr 9, 2025
4940e0c
test: avoid flakiness
pmarchini Apr 9, 2025
740e7b3
test: move fixture
pmarchini Apr 9, 2025
824b8e9
test: specify SIGINT
pmarchini Apr 9, 2025
c4ce83b
test: add isolation option to global hooks configuration
pmarchini Apr 9, 2025
ecb250a
test: add assertions for detailed error
pmarchini Apr 9, 2025
29b6ba5
test: use IPC for graceful child process exit in global setup
pmarchini Apr 9, 2025
287bf50
test: capture stderr for detailed error
pmarchini Apr 9, 2025
9f28b2c
test_runner: handle process exit events in harness and global setup
pmarchini Apr 9, 2025
0ae1c09
fixup! test_runner: handle process exit events in harness and global …
pmarchini Apr 9, 2025
46a2340
test_runner: remove exit event listeners in harness
pmarchini Apr 9, 2025
44b4eb0
test: forward process envs
pmarchini Apr 10, 2025
4a613be
test_runner: remove shared context between global setup and teardown
pmarchini Apr 13, 2025
19ca427
test_runner: update documentation
pmarchini Apr 13, 2025
b4c1c12
test_runner: remove fixture
pmarchini Apr 14, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2452,6 +2452,19 @@ added:
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.

### `--test-global-setup=module`

<!-- YAML
added: REPLACEME
-->

> Stability: 1.0 - Early development

Specify a module that will be evaluated before all tests are executed and
can be used to setup global state or fixtures for tests.

See the documentation on [global setup and teardown][] for more details.

### `--test-isolation=mode`

<!-- YAML
Expand Down Expand Up @@ -3347,6 +3360,7 @@ one is included in the list below.
* `--test-coverage-functions`
* `--test-coverage-include`
* `--test-coverage-lines`
* `--test-global-setup`
* `--test-isolation`
* `--test-name-pattern`
* `--test-only`
Expand Down Expand Up @@ -3898,6 +3912,7 @@ node --stack-trace-limit=12 -p -e "Error.stackTraceLimit" # prints 12
[emit_warning]: process.md#processemitwarningwarning-options
[environment_variables]: #environment-variables
[filtering tests by name]: test.md#filtering-tests-by-name
[global setup and teardown]: test.md#global-setup-and-teardown
[jitless]: https://v8.dev/blog/jitless
[libuv threadpool documentation]: https://docs.libuv.org/en/latest/threadpool.html
[module compile cache]: module.md#module-compile-cache
Expand Down
54 changes: 54 additions & 0 deletions doc/api/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -397,6 +397,60 @@ their dependencies. When a change is detected, the test runner will
rerun the tests affected by the change.
The test runner will continue to run until the process is terminated.

## Global setup and teardown

<!-- YAML
added: REPLACEME
-->

> Stability: 1.0 - Early development

The test runner supports specifying a module that will be evaluated before all tests are executed and
can be used to setup global state or fixtures for tests. This is useful for preparing resources or setting up
shared state that is required by multiple tests.

This module can export any of the following:

* A `globalSetup` function which runs once before all tests start
* A `globalTeardown` function which runs once after all tests complete

The module is specified using the `--test-global-setup` flag when running tests from the command line.

```cjs
// setup-module.js
async function globalSetup() {
// Setup shared resources, state, or environment
console.log('Global setup executed');
// Run servers, create files, prepare databases, etc.
}

async function globalTeardown() {
// Clean up resources, state, or environment
console.log('Global teardown executed');
// Close servers, remove files, disconnect from databases, etc.
}

module.exports = { globalSetup, globalTeardown };
```

```mjs
// setup-module.mjs
export async function globalSetup() {
// Setup shared resources, state, or environment
console.log('Global setup executed');
// Run servers, create files, prepare databases, etc.
}

export async function globalTeardown() {
// Clean up resources, state, or environment
console.log('Global teardown executed');
// Close servers, remove files, disconnect from databases, etc.
}
```

If the global setup function throws an error, no tests will be run and the process will exit with a non-zero exit code.
The global teardown function will not be called in this case.

## Running tests from the command line

The Node.js test runner can be invoked from the command line by passing the
Expand Down
3 changes: 3 additions & 0 deletions doc/node-config-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -392,6 +392,9 @@
"test-coverage-lines": {
"type": "number"
},
"test-global-setup": {
"type": "string"
},
"test-isolation": {
"type": "string"
},
Expand Down
3 changes: 3 additions & 0 deletions doc/node.1
Original file line number Diff line number Diff line change
Expand Up @@ -464,6 +464,9 @@ Require a minimum threshold for line coverage (0 - 100).
Configures the test runner to exit the process once all known tests have
finished executing even if the event loop would otherwise remain active.
.
.It Fl -test-global-setup
Specifies a module containing global setup and teardown functions for the test runner.
.
.It Fl -test-isolation Ns = Ns Ar mode
Configures the type of test isolation used in the test runner.
.
Expand Down
32 changes: 28 additions & 4 deletions lib/internal/test_runner/harness.js
Original file line number Diff line number Diff line change
Expand Up @@ -25,29 +25,29 @@ const {
parseCommandLine,
reporterScope,
shouldColorizeTestFiles,
setupGlobalSetupTeardownFunctions,
} = require('internal/test_runner/utils');
const { queueMicrotask } = require('internal/process/task_queues');
const { TIMEOUT_MAX } = require('internal/timers');
const { clearInterval, setInterval } = require('timers');
const { bigint: hrtime } = process.hrtime;
const resolvedPromise = PromiseResolve();
const testResources = new SafeMap();
let globalRoot;
let globalSetupExecuted = false;

testResources.set(reporterScope.asyncId(), reporterScope);

function createTestTree(rootTestOptions, globalOptions) {
const buildPhaseDeferred = PromiseWithResolvers();
const isFilteringByName = globalOptions.testNamePatterns ||
globalOptions.testSkipPatterns;
globalOptions.testSkipPatterns;
const isFilteringByOnly = (globalOptions.isolation === 'process' || process.env.NODE_TEST_CONTEXT) ?
globalOptions.only : true;
const harness = {
__proto__: null,
buildPromise: buildPhaseDeferred.promise,
buildSuites: [],
isWaitingForBuildPhase: false,
bootstrapPromise: resolvedPromise,
watching: false,
config: globalOptions,
coverage: null,
Expand All @@ -71,6 +71,21 @@ function createTestTree(rootTestOptions, globalOptions) {
snapshotManager: null,
isFilteringByName,
isFilteringByOnly,
async runBootstrap() {
if (globalSetupExecuted) {
return PromiseResolve();
}
globalSetupExecuted = true;
const globalSetupFunctions = await setupGlobalSetupTeardownFunctions(
globalOptions.globalSetupPath,
globalOptions.cwd,
);
harness.globalTeardownFunction = globalSetupFunctions.globalTeardownFunction;
if (typeof globalSetupFunctions.globalSetupFunction === 'function') {
return globalSetupFunctions.globalSetupFunction();
}
return PromiseResolve();
},
async waitForBuildPhase() {
if (harness.buildSuites.length > 0) {
await SafePromiseAllReturnVoid(harness.buildSuites);
Expand All @@ -81,6 +96,7 @@ function createTestTree(rootTestOptions, globalOptions) {
};

harness.resetCounters();
harness.bootstrapPromise = harness.runBootstrap();
globalRoot = new Test({
__proto__: null,
...rootTestOptions,
Expand Down Expand Up @@ -232,6 +248,11 @@ function setupProcessState(root, globalOptions) {
'Promise resolution is still pending but the event loop has already resolved',
kCancelledByParent));

if (root.harness.globalTeardownFunction) {
await root.harness.globalTeardownFunction();
root.harness.globalTeardownFunction = null;
}

hook.disable();
process.removeListener('uncaughtException', exceptionHandler);
process.removeListener('unhandledRejection', rejectionHandler);
Expand Down Expand Up @@ -278,7 +299,10 @@ function lazyBootstrapRoot() {
process.exitCode = kGenericUserError;
}
});
globalRoot.harness.bootstrapPromise = globalOptions.setup(globalRoot.reporter);
globalRoot.harness.bootstrapPromise = SafePromiseAllReturnVoid([
globalRoot.harness.bootstrapPromise,
globalOptions.setup(globalRoot.reporter),
]);
}
return globalRoot;
}
Expand Down
16 changes: 14 additions & 2 deletions lib/internal/test_runner/runner.js
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ const {
} = require('internal/test_runner/utils');
const { Glob } = require('internal/fs/glob');
const { once } = require('events');
const { validatePath } = require('internal/fs/utils');
const {
triggerUncaughtException,
exitCodes: { kGenericUserError },
Expand Down Expand Up @@ -556,6 +557,7 @@ function run(options = kEmptyObject) {
isolation = 'process',
watch,
setup,
globalSetupPath,
only,
globPatterns,
coverage = false,
Expand Down Expand Up @@ -665,6 +667,10 @@ function run(options = kEmptyObject) {
validateStringArray(argv, 'options.argv');
validateStringArray(execArgv, 'options.execArgv');

if (globalSetupPath != null) {
validatePath(globalSetupPath, 'options.globalSetupPath');
}

const rootTestOptions = { __proto__: null, concurrency, timeout, signal };
const globalOptions = {
__proto__: null,
Expand All @@ -679,6 +685,7 @@ function run(options = kEmptyObject) {
branchCoverage: branchCoverage,
functionCoverage: functionCoverage,
cwd,
globalSetupPath,
};
const root = createTestTree(rootTestOptions, globalOptions);
let testFiles = files ?? createTestFileList(globPatterns, cwd);
Expand Down Expand Up @@ -751,7 +758,9 @@ function run(options = kEmptyObject) {
const cascadedLoader = esmLoader.getOrInitializeCascadedLoader();
let topLevelTestCount = 0;

root.harness.bootstrapPromise = promise;
root.harness.bootstrapPromise = root.harness.bootstrapPromise ?
SafePromiseAllReturnVoid([root.harness.bootstrapPromise, promise]) :
promise;

const userImports = getOptionValue('--import');
for (let i = 0; i < userImports.length; i++) {
Expand Down Expand Up @@ -796,12 +805,15 @@ function run(options = kEmptyObject) {
debug('beginning test execution');
root.entryFile = null;
finishBootstrap();
root.processPendingSubtests();
return root.processPendingSubtests();
};
}
}

const runChain = async () => {
if (root.harness?.bootstrapPromise) {
await root.harness.bootstrapPromise;
}
if (typeof setup === 'function') {
await setup(root.reporter);
}
Expand Down
34 changes: 32 additions & 2 deletions lib/internal/test_runner/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ const {
} = primordials;

const { AsyncResource } = require('async_hooks');
const { relative, sep } = require('path');
const { relative, sep, resolve } = require('path');
const { createWriteStream } = require('fs');
const { pathToFileURL } = require('internal/url');
const { getOptionValue } = require('internal/options');
Expand All @@ -41,7 +41,12 @@ const {
kIsNodeError,
} = require('internal/errors');
const { compose } = require('stream');
const { validateInteger } = require('internal/validators');
const {
validateInteger,
validateFunction,
} = require('internal/validators');
const { validatePath } = require('internal/fs/utils');
const { kEmptyObject } = require('internal/util');

const coverageColors = {
__proto__: null,
Expand Down Expand Up @@ -199,6 +204,7 @@ function parseCommandLine() {
const timeout = getOptionValue('--test-timeout') || Infinity;
const isChildProcess = process.env.NODE_TEST_CONTEXT === 'child';
const isChildProcessV8 = process.env.NODE_TEST_CONTEXT === 'child-v8';
let globalSetupPath;
let concurrency;
let coverageExcludeGlobs;
let coverageIncludeGlobs;
Expand All @@ -223,6 +229,7 @@ function parseCommandLine() {
} else {
destinations = getOptionValue('--test-reporter-destination');
reporters = getOptionValue('--test-reporter');
globalSetupPath = getOptionValue('--test-global-setup');
if (reporters.length === 0 && destinations.length === 0) {
ArrayPrototypePush(reporters, kDefaultReporter);
}
Expand Down Expand Up @@ -328,6 +335,7 @@ function parseCommandLine() {
only,
reporters,
setup,
globalSetupPath,
shard,
sourceMaps,
testNamePatterns,
Expand Down Expand Up @@ -597,6 +605,27 @@ function getCoverageReport(pad, summary, symbol, color, table) {
return report;
}

async function setupGlobalSetupTeardownFunctions(globalSetupPath, cwd) {
let globalSetupFunction;
let globalTeardownFunction;
if (globalSetupPath) {
validatePath(globalSetupPath, 'options.globalSetupPath');
const fileURL = pathToFileURL(resolve(cwd, globalSetupPath));
const cascadedLoader = require('internal/modules/esm/loader').getOrInitializeCascadedLoader();
const globalSetupModule = await cascadedLoader
.import(fileURL, pathToFileURL(cwd + sep).href, kEmptyObject);
if (globalSetupModule.globalSetup) {
validateFunction(globalSetupModule.globalSetup, 'globalSetupModule.globalSetup');
globalSetupFunction = globalSetupModule.globalSetup;
}
if (globalSetupModule.globalTeardown) {
validateFunction(globalSetupModule.globalTeardown, 'globalSetupModule.globalTeardown');
globalTeardownFunction = globalSetupModule.globalTeardown;
}
}
return { __proto__: null, globalSetupFunction, globalTeardownFunction };
}

module.exports = {
convertStringToRegExp,
countCompletedTest,
Expand All @@ -607,4 +636,5 @@ module.exports = {
reporterScope,
shouldColorizeTestFiles,
getCoverageReport,
setupGlobalSetupTeardownFunctions,
};
4 changes: 4 additions & 0 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -761,6 +761,10 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
"exclude files from coverage report that match this glob pattern",
&EnvironmentOptions::coverage_exclude_pattern,
kAllowedInEnvvar);
AddOption("--test-global-setup",
"specifies the path to the global setup file",
&EnvironmentOptions::test_global_setup_path,
kAllowedInEnvvar);
AddOption("--test-udp-no-try-send", "", // For testing only.
&EnvironmentOptions::test_udp_no_try_send);
AddOption("--throw-deprecation",
Expand Down
1 change: 1 addition & 0 deletions src/node_options.h
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,7 @@ class EnvironmentOptions : public Options {
std::vector<std::string> test_name_pattern;
std::vector<std::string> test_reporter;
std::vector<std::string> test_reporter_destination;
std::string test_global_setup_path;
bool test_only = false;
bool test_udp_no_try_send = false;
std::string test_isolation = "process";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
'use strict';

const test = require('node:test');
const assert = require('node:assert');
const fs = require('node:fs');

test('Another test that verifies setup flag existance', (t) => {
const setupFlagPath = process.env.SETUP_FLAG_PATH;
assert.ok(fs.existsSync(setupFlagPath), 'Setup flag file should exist');

const content = fs.readFileSync(setupFlagPath, 'utf8');
assert.strictEqual(content, 'Setup was executed');
});
Loading
Loading