Skip to content

Commit fd3abf8

Browse files
authored
destination-vectara: [autopull] base image + poetry + up_to_date (#38432)
1 parent ac1e393 commit fd3abf8

File tree

4 files changed

+42
-9
lines changed

4 files changed

+42
-9
lines changed

airbyte-integrations/connectors/destination-vectara/README.md

+38-6
Original file line numberDiff line numberDiff line change
@@ -78,8 +78,44 @@ async def post_connector_install(connector_container: Container) -> Container:
7878
return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")
7979
```
8080

81-
#### Build your own connector image
8281

82+
83+
#### Use `airbyte-ci` to build your connector
84+
The Airbyte way of building this connector is to use our `airbyte-ci` tool.
85+
You can follow install instructions [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md#L1).
86+
Then running the following command will build your connector:
87+
88+
```bash
89+
airbyte-ci connectors --name destination-vectara build
90+
```
91+
Once the command is done, you will find your connector image in your local docker registry: `airbyte/destination-vectara:dev`.
92+
93+
##### Customizing our build process
94+
When contributing on our connector you might need to customize the build process to add a system dependency or set an env var.
95+
You can customize our build process by adding a `build_customization.py` module to your connector.
96+
This module should contain a `pre_connector_install` and `post_connector_install` async function that will mutate the base image and the connector container respectively.
97+
It will be imported at runtime by our build process and the functions will be called if they exist.
98+
99+
Here is an example of a `build_customization.py` module:
100+
```python
101+
from __future__ import annotations
102+
103+
from typing import TYPE_CHECKING
104+
105+
if TYPE_CHECKING:
106+
# Feel free to check the dagger documentation for more information on the Container object and its methods.
107+
# https://dagger-io.readthedocs.io/en/sdk-python-v0.6.4/
108+
from dagger import Container
109+
110+
111+
async def pre_connector_install(base_image_container: Container) -> Container:
112+
return await base_image_container.with_env_variable("MY_PRE_BUILD_ENV_VAR", "my_pre_build_env_var_value")
113+
114+
async def post_connector_install(connector_container: Container) -> Container:
115+
return await connector_container.with_env_variable("MY_POST_BUILD_ENV_VAR", "my_post_build_env_var_value")
116+
```
117+
118+
#### Build your own connector image
83119
This connector is built using our dynamic built process in `airbyte-ci`.
84120
The base image used to build it is defined within the metadata.yaml file under the `connectorBuildOptions`.
85121
The build logic is defined using [Dagger](https://dagger.io/) [here](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/pipelines/builds/python_connectors.py).
@@ -88,7 +124,6 @@ It does not rely on a Dockerfile.
88124
If you would like to patch our connector and build your own a simple approach would be to:
89125

90126
1. Create your own Dockerfile based on the latest version of the connector image.
91-
92127
```Dockerfile
93128
FROM airbyte/destination-vectara:latest
94129

@@ -99,17 +134,14 @@ RUN pip install ./airbyte/integration_code
99134
# ENV AIRBYTE_ENTRYPOINT "python /airbyte/integration_code/main.py"
100135
# ENTRYPOINT ["python", "/airbyte/integration_code/main.py"]
101136
```
102-
103137
Please use this as an example. This is not optimized.
104138

105139
2. Build your image:
106-
107140
```bash
108141
docker build -t airbyte/destination-vectara:dev .
109142
# Running the spec command against your patched connector
110143
docker run airbyte/destination-vectara:dev spec
111144
```
112-
113145
#### Run
114146

115147
Then run any of the connector commands as follows:
@@ -172,4 +204,4 @@ You've checked out the repo, implemented a million dollar feature, and you're re
172204
4. Make the connector documentation and its changelog is up to date (`docs/integrations/destinations/vectara.md`).
173205
5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention).
174206
6. Pat yourself on the back for being an awesome contributor.
175-
7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.
207+
7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master.

airbyte-integrations/connectors/destination-vectara/metadata.yaml

+2-2
Original file line numberDiff line numberDiff line change
@@ -9,11 +9,11 @@ data:
99
cloud:
1010
enabled: true
1111
connectorBuildOptions:
12-
baseImage: docker.io/airbyte/python-connector-base:1.1.0@sha256:bd98f6505c6764b1b5f99d3aedc23dfc9e9af631a62533f60eb32b1d3dbab20c
12+
baseImage: docker.io/airbyte/python-connector-base:1.2.0@sha256:c22a9d97464b69d6ef01898edf3f8612dc11614f05a84984451dde195f337db9
1313
connectorSubtype: database
1414
connectorType: destination
1515
definitionId: 102900e7-a236-4c94-83e4-a4189b99adc2
16-
dockerImageTag: 0.2.3
16+
dockerImageTag: 0.2.4
1717
dockerRepository: airbyte/destination-vectara
1818
githubIssueLabel: destination-vectara
1919
icon: vectara.svg

airbyte-integrations/connectors/destination-vectara/pyproject.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
44

55
[tool.poetry]
66
name = "airbyte-destination-vectara"
7-
version = "0.2.3"
7+
version = "0.2.4"
88
description = "Airbyte destination implementation for Vectara"
99
authors = ["Airbyte <[email protected]>"]
1010
license = "MIT"

docs/integrations/destinations/vectara.md

+1
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,7 @@ In addition, in the connector UI you define two set of fields for this connector
6565

6666
| Version | Date | Pull Request | Subject |
6767
| :------ | :--------- | :-------------------------------------------------------- | :----------------------------------------------------------- |
68+
| 0.2.4 | 2024-05-20 | [38432](https://github.com/airbytehq/airbyte/pull/38432) | [autopull] base image + poetry + up_to_date |
6869
| 0.2.3 | 2024-03-22 | [#37333](https://github.com/airbytehq/airbyte/pull/37333) | Updated CDK & pytest version to fix security vulnerabilities |
6970
| 0.2.2 | 2024-03-22 | [#36261](https://github.com/airbytehq/airbyte/pull/36261) | Move project to Poetry |
7071
| 0.2.1 | 2024-03-05 | [#35206](https://github.com/airbytehq/airbyte/pull/35206) | Fix: improved title parsing |

0 commit comments

Comments
 (0)