Skip to content

Commit 04136bb

Browse files
committed
docs(live-test): Update Readme
1 parent 37dfcd4 commit 04136bb

File tree

1 file changed

+33
-8
lines changed

1 file changed

+33
-8
lines changed

airbyte-ci/connectors/live-tests/README.md

+33-8
Original file line numberDiff line numberDiff line change
@@ -63,7 +63,7 @@ poetry run live-tests debug read \
6363
--catalog-path=configured_catalog.json
6464
```
6565

66-
It will store the results in a `live_test_debug_reports` directory under the current working directory:
66+
It will store the results in a `live_test_debug_reports` directory under the current working directory:
6767

6868
```
6969
live_tests_debug_reports
@@ -110,19 +110,26 @@ mitmweb --rfile=http_dump.mitm
110110
```
111111

112112
## Regression tests
113-
We created a regression test suite to run tests to compare the outputs of connector commands on different versions of the same connector.
113+
We created a regression test suite to run tests to compare the outputs of connector commands on different versions of the same connector.
114+
115+
## Tutorial(s)
116+
* [Loom Walkthrough (Airbyte Only)](https://www.loom.com/share/97c49d7818664b119cff6911a8a211a2?sid=4570a5b6-9c81-4db3-ba33-c74dc5845c3c)
117+
* [Internal Docs (Airbyte Only)](https://docs.google.com/document/d/1pzTxJTsooc9iQDlALjvOWtnq6yRTvzVtbkJxY4R36_I/edit)
118+
119+
### How to Use
120+
121+
> ⚠️ **Note:** While you can use this tool without building a dev image, to achieve your goals you will likely need to have installed [airbyte-ci](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) and know how to build a dev image.
114122
115123
You can run the existing test suites with the following command:
116124

117125
#### With local connection objects (`config.json`, `catalog.json`, `state.json`)
118126
```bash
119-
poetry run pytest src/live_tests/regression_tests \
127+
poetry run pytest src/live_tests/regression_tests \
120128
--connector-image=airbyte/source-faker \
121-
--connection-id=<CONNECTION-ID> \
122129
--config-path=<path-to-config-path> \
123130
--catalog-path=<path-to-catalog-path> \
124131
--target-version=dev \
125-
--control-version=latest
132+
--control-version=latest \
126133
--pr-url=<PR-URL> # The URL of the PR you are testing
127134
```
128135

@@ -132,9 +139,8 @@ The live connection objects will be fetched.
132139
```bash
133140
poetry run pytest src/live_tests/regression_tests \
134141
--connector-image=airbyte/source-faker \
135-
--connection-id=<CONNECTION-ID> \
136142
--target-version=dev \
137-
--control-version=latest
143+
--control-version=latest \
138144
--pr-url=<PR-URL> # The URL of the PR you are testing
139145
```
140146

@@ -235,6 +241,25 @@ The test suite run will produce test artifacts in the `/tmp/regression_tests_art
235241
We use a containerized `mitmproxy` to capture the HTTP traffic between the connector and the source. Connector command runs produce `http_dump.mitm` (can be consumed with `mitmproxy` (version `>=10`) for debugging) and `http_dump.har` (a JSON encoded version of the mitm dump) artifacts.
236242
The traffic recorded on the control connector is passed to the target connector proxy to cache the responses for requests with the same URL. This is useful to avoid hitting the source API multiple times when running the same command on different versions of the connector.
237243

244+
### Custom CLI Arguments
245+
246+
| Argument | Description | Required/Optional |
247+
|----------------------------|---------------------------------------------------------------------------------------------------------------------------|-------------------|
248+
| `--connector-image` | Docker image name of the connector to debug (e.g., `airbyte/source-faker:latest`, `airbyte/source-faker:dev`). | Required |
249+
| `--control-version` | Version of the control connector for regression testing. | Required |
250+
| `--target-version` | Version of the connector being tested. (Defaults to dev) | Optional |
251+
| `--pr-url` | URL of the pull request being tested. | Required |
252+
| `--connection-id` | ID of the connection for live testing. If not provided, a prompt will appear to choose. | Optional |
253+
| `--config-path` | Path to the custom source configuration file. | Optional |
254+
| `--catalog-path` | Path to the custom configured catalog file. | Optional |
255+
| `--state-path` | Path to the custom state file. | Optional |
256+
| `--http-cache` | Use the HTTP cache for the connector. | Optional |
257+
| `--run-id` | Unique identifier for the test run. If not provided, a timestamp will be used. | Optional |
258+
| `--auto-select-connection` | Automatically select a connection for testing. | Optional |
259+
| `--stream` | Name of the stream to test. Can be specified multiple times to test multiple streams. | Optional |
260+
| `--should-read-with-state` | Specify whether to read with state. If not provided, a prompt will appear to choose. | Optional |
261+
262+
238263
## Changelog
239264

240265
### 0.16.0
@@ -299,7 +324,7 @@ Modify diff output for `discover` and `read` tests.
299324

300325
### 0.5.1
301326
Handle connector command execution errors.
302-
327+
303328
### 0.5.0
304329
Add new tests and confirmation prompts.
305330

0 commit comments

Comments
 (0)