Skip to content

ci(NODE-6686): deployed Atlas cluster tests use secrets manager #4384

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 30, 2025
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .evergreen/config.in.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1602,9 +1602,11 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
env:
MONGODB_VERSION: "7.0"
LAMBDA_STACK_NAME: dbx-node-lambda
CLUSTER_PREFIX: dbx-node-lambda
VAULT_NAME: atlas
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/setup-atlas-cluster.sh
- command: expansions.update
Expand All @@ -1615,7 +1617,6 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/teardown-atlas-cluster.sh
setup_group_can_fail_task: true
Expand All @@ -1630,9 +1631,9 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
env:
MONGODB_VERSION: "7.0"
CLUSTER_PREFIX: dbx-node-search-indexes
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/setup-atlas-cluster.sh
- command: expansions.update
Expand All @@ -1643,7 +1644,6 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/teardown-atlas-cluster.sh
setup_group_can_fail_task: true
Expand Down
8 changes: 4 additions & 4 deletions .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4623,9 +4623,11 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
env:
MONGODB_VERSION: '7.0'
LAMBDA_STACK_NAME: dbx-node-lambda
CLUSTER_PREFIX: dbx-node-lambda
VAULT_NAME: atlas
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/setup-atlas-cluster.sh
- command: expansions.update
Expand All @@ -4636,7 +4638,6 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/teardown-atlas-cluster.sh
setup_group_can_fail_task: true
Expand All @@ -4650,9 +4651,9 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
env:
MONGODB_VERSION: '7.0'
CLUSTER_PREFIX: dbx-node-search-indexes
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/setup-atlas-cluster.sh
- command: expansions.update
Expand All @@ -4663,7 +4664,6 @@ task_groups:
params:
working_dir: src
binary: bash
add_expansions_to_env: true
args:
- ${DRIVERS_TOOLS}/.evergreen/atlas/teardown-atlas-cluster.sh
setup_group_can_fail_task: true
Expand Down
110 changes: 69 additions & 41 deletions test/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,18 +26,23 @@ about the types of tests and how to run them.
- [Skipping Tests](#skipping-tests)
- [Running Benchmarks](#running-benchmarks)
- [Configuration](#configuration)
- [Secrets](#secrets)
- [Testing with Special Environments](#testing-with-special-environments)
- [Serverless](#serverless)
- [Load Balanced](#load-balanced)
- [Client-Side Field-Level Encryption (CSFLE)](#client-side-field-level-encryption-csfle)
- [Testing driver changes with mongosh](#testing-driver-changes-with-mongosh)
- [Point mongosh to the driver](#point-mongosh-to-the-driver)
- [Run specific package tests](#run-specific-package-tests)
- [KMIP FLE support tests](#kmip-fle-support-tests)
- [Deployed KMS Tests](#deployed-kms-tests)
- [Azure KMS](#azure-kms)
- [GCP KMS](#gcp-kms)
- [Deployed Atlas Tests](#deployed-atlas-tests)
- [Launching an Atlas Cluster](#launching-an-atlas-cluster)
- [Search Indexes](#search-indexes)
- [Deployed Lambda Tests](#deployed-lambda-tests)
- [TODO Special Env Sections](#todo-special-env-sections)
- [Testing driver changes with mongosh](#testing-driver-changes-with-mongosh)
- [Point mongosh to the driver](#point-mongosh-to-the-driver)
- [Run specific package tests](#run-specific-package-tests)

## About the Tests

Expand Down Expand Up @@ -594,44 +599,6 @@ The following steps will walk you through how to run the tests for CSFLE.

To run the functional tests using the crypt shared library instead of `mongocryptd`, download the appropriate version of the crypt shared library for the enterprise server version [here](https://www.mongodb.com/download-center/enterprise/releases) and then set the location of it in the environment variable `CRYPT_SHARED_LIB_PATH`.

#### Testing driver changes with mongosh

These steps require `mongosh` to be available locally. Clone it from GitHub.

`mongosh` uses a `lerna` monorepo. As a result, `mongosh` contains multiple references to the `mongodb` package
in their `package.json`s.

Set up `mongosh` by following the steps in the `mongosh` readme.

##### Point mongosh to the driver

mongosh contains a script that does this. To use the script, create an environment
variable `REPLACE_PACKAGE` that contains a string in the form
`mongodb:<path to your local instance of the driver>`. The package replacement script will replace
all occurrences of `mongodb` with the local path of your driver.

An alternative, which can be useful for
testing a release, is to first run `npm pack` on the driver. This generates a tarball containing all the code
that would be uploaded to `npm` if it were released. Then, set the environment variable `REPLACE_PACKAGE`
with the full path to the file.

Once the environment variable is set, run replace package in `mongosh` with:
```sh
npm run replace:package
```

##### Run specific package tests

`mongosh`'s readme documents how to run its tests. Most likely, it isn't necessary to run all of mongosh's
tests. The `mongosh` readme also documents how to run tests for a particular scope. The scopes are
listed in the `generate_mongosh_tasks.js` evergreen generation script.

For example, to run the `service-provider-server` package, run the following command in `mongosh`:

```shell
lerna run test --scope @mongosh/service-provider-server
```

#### KMIP FLE support tests

1. Install `virtualenv`:
Expand Down Expand Up @@ -729,6 +696,29 @@ source $DRIVERS_TOOLS/.evergreen/init-node-and-npm-env.sh
bash .evergreen/run-deployed-gcp-kms-tests.sh
```


### Deployed Atlas Tests

#### Launching an Atlas Cluster

Using drivers evergreen tools, run the `setup-atlas-cluster` script. You must also set the CLUSTER_PREFIX environment variable.

```bash
CLUSTER_PREFIX=dbx-node-lambda bash ${DRIVERS_TOOLS}/.evergreen/atlas/setup-atlas-cluster.sh
```

The URI of the cluster is available in the `atlas-expansions.yml` file.

#### Search Indexes

1. Set up an Atlas cluster, as outlined in the "Launching an Atlas Cluster" section.
2. Add the URI of the cluster to the environment as the MONGODB_URI environment variable.
3. Run the tests with `npm run check:search-indexes`.

#### Deployed Lambda Tests

TODO

### TODO Special Env Sections

- Kerberos
Expand All @@ -755,3 +745,41 @@ bash .evergreen/run-deployed-gcp-kms-tests.sh
[npm-csfle]: https://www.npmjs.com/package/mongodb-client-encryption
[atlas-api-key]: https://docs.atlas.mongodb.com/tutorial/configure-api-access/organization/create-one-api-key
[scram-auth]: https://docs.atlas.mongodb.com/security-add-mongodb-users/#database-user-authentication

## Testing driver changes with mongosh

These steps require `mongosh` to be available locally. Clone it from GitHub.

`mongosh` uses a `lerna` monorepo. As a result, `mongosh` contains multiple references to the `mongodb` package
in their `package.json`s.

Set up `mongosh` by following the steps in the `mongosh` readme.

### Point mongosh to the driver

mongosh contains a script that does this. To use the script, create an environment
variable `REPLACE_PACKAGE` that contains a string in the form
`mongodb:<path to your local instance of the driver>`. The package replacement script will replace
all occurrences of `mongodb` with the local path of your driver.

An alternative, which can be useful for
testing a release, is to first run `npm pack` on the driver. This generates a tarball containing all the code
that would be uploaded to `npm` if it were released. Then, set the environment variable `REPLACE_PACKAGE`
with the full path to the file.

Once the environment variable is set, run replace package in `mongosh` with:
```sh
npm run replace:package
```

### Run specific package tests

`mongosh`'s readme documents how to run its tests. Most likely, it isn't necessary to run all of mongosh's
tests. The `mongosh` readme also documents how to run tests for a particular scope. The scopes are
listed in the `generate_mongosh_tasks.js` evergreen generation script.

For example, to run the `service-provider-server` package, run the following command in `mongosh`:

```shell
lerna run test --scope @mongosh/service-provider-server
```