You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
#### Build & Activate Virtual Environment and install dependencies
12
-
From this connector directory, create a virtual environment:
13
-
```
14
-
python -m venv .venv
15
-
```
16
13
17
-
This will generate a virtualenv for this module in `.venv/`. Make sure this venv is active in your
18
-
development environment of choice. To activate it from the terminal, run:
19
-
```
20
-
source .venv/bin/activate
21
-
pip install -r requirements.txt
14
+
### Installing the connector
15
+
From this connector directory, run:
16
+
```bash
17
+
poetry install --with dev
22
18
```
23
-
If you are in an IDE, follow your IDE's instructions to activate the virtualenv.
24
19
25
-
Note that while we are installing dependencies from `requirements.txt`, you should only edit `setup.py` for your dependencies. `requirements.txt` is
26
-
used for editable installs (`pip install -e`) to pull in Python dependencies from the monorepo and will call `setup.py`.
27
-
If this is mumbo jumbo to you, don't worry about it, just put your deps in `setup.py` but install using `pip install -r requirements.txt` and everything
28
-
should work as you expect.
29
20
30
-
####Create credentials
31
-
**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.io/integrations/sources/google-ads)
32
-
to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_google_ads/spec.json` file.
21
+
### Create credentials
22
+
**If you are a community contributor**, follow the instructions in the [documentation](https://docs.airbyte.com/integrations/sources/google-ads)
23
+
to generate the necessary credentials. Then create a file `secrets/config.json` conforming to the `source_google_ads/spec.yaml` file.
33
24
Note that any directory named `secrets` is gitignored across the entire Airbyte repo, so there is no danger of accidentally checking in sensitive information.
34
-
See `integration_tests/sample_config.json` for a sample config file.
25
+
See `sample_files/sample_config.json` for a sample config file.
35
26
36
-
**If you are an Airbyte core member**, copy the credentials in Lastpass under the secret name `source google-ads test creds`
Once the command is done, you will find your connector image in your local docker registry: `airbyte/source-google-ads:dev`.
60
-
61
-
##### Customizing our build process
62
-
When contributing on our connector you might need to customize the build process to add a system dependency or set an env var.
63
-
You can customize our build process by adding a `build_customization.py` module to your connector.
64
-
This module should contain a `pre_connector_install` and `post_connector_install` async function that will mutate the base image and the connector container respectively.
65
-
It will be imported at runtime by our build process and the functions will be called if they exist.
66
-
67
-
Here is an example of a `build_customization.py` module:
68
-
```python
69
-
from__future__import annotations
70
-
71
-
from typing importTYPE_CHECKING
72
-
73
-
ifTYPE_CHECKING:
74
-
# Feel free to check the dagger documentation for more information on the Container object and its methods.
This connector is built using our dynamic built process in `airbyte-ci`.
88
-
The base image used to build it is defined within the metadata.yaml file under the `connectorBuildOptions`.
89
-
The build logic is defined using [Dagger](https://dagger.io/)[here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/pipelines/builds/python_connectors.py).
90
-
It does not rely on a Dockerfile.
91
-
92
-
If you would like to patch our connector and build your own a simple approach would be to:
93
-
94
-
1. Create your own Dockerfile based on the latest version of the connector image.
95
-
```Dockerfile
96
-
FROM airbyte/source-google-ads:latest
97
48
98
-
COPY . ./airbyte/integration_code
99
-
RUN pip install ./airbyte/integration_code
49
+
An image will be available on your host with the tag `airbyte/source-google-ads:dev`.
100
50
101
-
# The entrypoint and default env vars are already set in the base image
You can run our full test suite locally using [`airbyte-ci`](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md):
124
63
```bash
125
64
airbyte-ci connectors --name=source-google-ads test
126
65
```
127
66
128
67
### Customizing acceptance Tests
129
-
Customize `acceptance-test-config.yml` file to configure tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
68
+
Customize `acceptance-test-config.yml` file to configure acceptance tests. See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference) for more information.
130
69
If your connector requires to create or destroy resources for use during acceptance tests create fixtures for it and place them inside integration_tests/acceptance.py.
131
70
132
-
## Dependency Management
133
-
All of your dependencies should go in `setup.py`, NOT `requirements.txt`. The requirements file is only used to connect internal Airbyte dependencies in the monorepo for local development.
134
-
We split dependencies between two groups, dependencies that are:
135
-
* required for your connector to work need to go to `MAIN_REQUIREMENTS` list.
136
-
* required for the testing need to go to `TEST_REQUIREMENTS` list
71
+
### Dependency Management
72
+
All of your dependencies should be managed via Poetry.
73
+
To add a new dependency, run:
74
+
```bash
75
+
poetry add <package-name>
76
+
```
77
+
78
+
Please commit the changes to `pyproject.toml` and `poetry.lock` files.
137
79
138
-
###Publishing a new version of the connector
80
+
## Publishing a new version of the connector
139
81
You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what?
140
82
1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=source-google-ads test`
141
-
2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors).
83
+
2. Bump the connector version (please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors)):
84
+
- bump the `dockerImageTag` value in in `metadata.yaml`
85
+
- bump the `version` value in `pyproject.toml`
142
86
3. Make sure the `metadata.yaml` content is up to date.
143
-
4. Make the connector documentation and its changelog is up to date (`docs/integrations/sources/google-ads.md`).
87
+
4. Make sure the connector documentation and its changelog is up to date (`docs/integrations/sources/google-ads.md`).
144
88
5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
145
89
6. Pat yourself on the back for being an awesome contributor.
146
90
7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
147
-
91
+
8. Once your PR is merged, the new version of the connector will be automatically published to Docker Hub and our connector registry.
0 commit comments