Skip to content

Declarative connectors runtime: Extend CDK publish process to update docker base image #22442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
flash1293 opened this issue Feb 6, 2023 · 1 comment
Closed

Declarative connectors runtime: Extend CDK publish process to update docker base image #22442

flash1293 opened this issue Feb 6, 2023 · 1 comment
Assignees
@maxi297
Labels
team/extensibility

Comments

@flash1293
Copy link
Contributor

flash1293 commented Feb 6, 2023 •
edited by sherifnada
Loading

When publishing a new version of the CDK, the docker base image implemented as part of #22438 should be re-built with the new version and published as well.

Acceptance criteria

  • There is a process in place that ensures there is a version of the airbyte/source-declarative-manifest docker image for every released CDK version with matching version numbers
  • Docker image should only publish if publish to PyPI also succeeded

Implementation Hints

  1. there is a script tools/integrartions/manage.sh which builds & publishes docker images for connectos. we can reuse that.
  2. we might also need to modify the CI .github yaml files to push that image
@maxi297
Copy link
Contributor

maxi297 commented Feb 9, 2023

https://github.com/airbytehq/github-workflow-test-repo-base repo can be used to perform some tests

@flash1293 flash1293 mentioned this issue Feb 23, 2023
Merged
@maxi297 maxi297 mentioned this issue Feb 27, 2023
Merged
maxi297 added a commit that referenced this issue Feb 27, 2023
@maxi297
[ISSUE #22442] test publish docker
c53bf35
maxi297 added a commit that referenced this issue Feb 27, 2023
@maxi297
[ISSUE #22442] test seems to be successful but still error
feecca0
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] could not build airbyte/integration-base-java:dev --pl…
70f08f9 
…atform linux/arm64
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] solving apt-get issue
4eef00f
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] adding sentry
050fea6
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] fixing sentry issue
154d4fd
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] integrating everything to the pipeline
1ebbc74
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] adding comment on why we use EC2 runners
ed07f2f
maxi297 added a commit that referenced this issue Feb 28, 2023
@maxi297
[ISSUE #22442] removing Sentry from publishing the Docker image
59fa9d6
maxi297 added a commit that referenced this issue Mar 1, 2023
@maxi297
[ISSUE #22442] publish docker (#23501)
34f6f63 
* [ISSUE #22442] test publish docker

* [ISSUE #22442] test seems to be successful but still error

* [ISSUE #22442] could not build airbyte/integration-base-java:dev --platform linux/arm64

* [ISSUE #22442] solving apt-get issue

* [ISSUE #22442] adding sentry

* [ISSUE #22442] fixing sentry issue

* [ISSUE #22442] integrating everything to the pipeline

* [ISSUE #22442] adding comment on why we use EC2 runners

* [ISSUE #22442] removing Sentry from publishing the Docker image

---------

Co-authored-by: Joe Reuter <[email protected]>
@maxi297 maxi297 closed this as completed Mar 1, 2023
maxi297 added a commit that referenced this issue Mar 1, 2023
@maxi297
[ISSUE #22442] move success message from pypi publish to docker image…
069bde0 
… publish
maxi297 added a commit that referenced this issue Mar 1, 2023
@maxi297
[ISSUE #22442] move success message from pypi publish to docker image…
38dbf23 
… publish
maxi297 added a commit that referenced this issue Mar 1, 2023
@maxi297
[ISSUE #22442] fix bump version
38e84f1
maxi297 added a commit that referenced this issue Mar 1, 2023
@maxi297
[ISSUE #22442] move success message from pypi publish to docker image… (
7376da4 
#23620)

* [ISSUE #22442] move success message from pypi publish to docker image publish

* [ISSUE #22442] move success message from pypi publish to docker image publish

* [ISSUE #22442] fix bump version
maxi297 added a commit that referenced this issue Mar 2, 2023
@maxi297
[ISSUE #22442] fixing race condition in the pipeline
33e2601
maxi297 added a commit that referenced this issue Mar 2, 2023
@maxi297
[ISSUE #22442] fixing race condition in the pipeline (#23670)
f1ee2c3
jbfbell pushed a commit that referenced this issue Mar 6, 2023
@maxi297 @jbfbell
[ISSUE #22442] fixing race condition in the pipeline (#23670)
b5ad20d
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this issue Mar 13, 2023
@maxi297 @danielduckworth
[ISSUE airbytehq#22442] publish docker (airbytehq#23501)
a76a544 
* [ISSUE airbytehq#22442] test publish docker

* [ISSUE airbytehq#22442] test seems to be successful but still error

* [ISSUE airbytehq#22442] could not build airbyte/integration-base-java:dev --platform linux/arm64

* [ISSUE airbytehq#22442] solving apt-get issue

* [ISSUE airbytehq#22442] adding sentry

* [ISSUE airbytehq#22442] fixing sentry issue

* [ISSUE airbytehq#22442] integrating everything to the pipeline

* [ISSUE airbytehq#22442] adding comment on why we use EC2 runners

* [ISSUE airbytehq#22442] removing Sentry from publishing the Docker image

---------

Co-authored-by: Joe Reuter <[email protected]>
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this issue Mar 13, 2023
@maxi297 @danielduckworth
[ISSUE airbytehq#22442] move success message from pypi publish to doc…
82f263e 
…ker image… (airbytehq#23620)

* [ISSUE airbytehq#22442] move success message from pypi publish to docker image publish

* [ISSUE airbytehq#22442] move success message from pypi publish to docker image publish

* [ISSUE airbytehq#22442] fix bump version
danielduckworth pushed a commit to danielduckworth/airbyte that referenced this issue Mar 13, 2023
@maxi297 @danielduckworth
[ISSUE airbytehq#22442] fixing race condition in the pipeline (airbyt…
35e43ae 
…ehq#23670)
@girarda girarda mentioned this issue Mar 15, 2023
Closed
@jbfbell jbfbell mentioned this issue Mar 15, 2023
Closed
37 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@maxi297 maxi297

Labels
team/extensibility
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@flash1293 @maxi297