Closed
Description
Tell us about the problem you're trying to solve
Now we force to write every new version to 4 different places:
airbyte-config/init/src/main/resources/config/STANDARD_SOURCE_DEFINITION/<connector_uuid>.json
airbyte-config/init/src/main/resources/seed/source_definitions.yaml
airbyte-integrations/connectors/<connector_name>/Dockerfile
airbyte-integrations/connectors/<connector_name>/setup.py
+
And update doc files:
docs/SUMMARY.md
docs/integrations/README.md
airbyte-integrations/builds.md
On my opinion it is tough for support and is the reason of merge conflicts.
Describe the solution you’d like
We can automate updating of these files during the publish
command.
- The source of truth should be one of the following variants:
- setup.py file. Certainly these options are available for Python packages but I'm sure that Java has a similar one.
For this we can use the following package options:
- name - <connector_unique_name>
- version - <connector_version>
--long-description - <connector_doc_filepath> or <docfile_full>
--classifiers - <connector_grade>
- Dockerfile. And It will be unified approach for all languages. For this we can use the following labels:
LABEL io.airbyte.name=airbyte/<connector_unique_name>
LABEL io.airbyte.version=<connector_version>
LABEL io.airbyte.grade=<connector_grade>
LABEL io.airbyte.docfile=<connector_doc_filepath>
- <connector_uuid> can be generated automatically for all new connectors(checking of the source_definitions.yaml file)
- <connector_doc_file_path> - should be saved into connectors' folders(e.g.:
airbyte-integrations/connectors/<connector_name>/<connector_name>.md
) And the publish command will create a symlink to the needed place. - <connector_grade> will be used for updating of
docs/integrations/README.md
- As result this CI job will add a new commit to a relevant branch if needed.
The main benefit is that all developer will support 2 files only (docfile and setup.py/Dockerfile)
And I'm ready to implement this.