Skip to content

fix pyyaml/cython dependency issue when building base-normalization #28451

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 19, 2023

Conversation

pedroslopez
Copy link
Contributor

What

Applies a workaround for yaml/pyyaml#601

This error is introduced by COPY --from=airbyte/base-airbyte-protocol-python:0.1.1 /airbyte /airbyte. When we get to installing dependencies in the dockerfile for this image here:

# Install python dependencies
WORKDIR /airbyte/base_python_structs
RUN pip install .

We error out with the cython_sources error.

airbyte/base-airbyte-protocol-python has only been published once, 2 years ago. It is unclear where this image comes from, what's in it, and how it was built. This isn't a great spot to be in, but we can workaround it for now with this fix until we know more.

@pedroslopez pedroslopez requested a review from a team as a code owner July 19, 2023 00:58
@pedroslopez pedroslopez requested a review from a team July 19, 2023 00:58
@pedroslopez pedroslopez enabled auto-merge (squash) July 19, 2023 01:02
@octavia-squidington-iii
Copy link
Collaborator

destination-mysql-strict-encrypt test report (commit ca28112095) - ❌

⏲️ Total pipeline duration: 30mn01s

Step Result
Validate airbyte-integrations/connectors/destination-mysql-strict-encrypt/metadata.yaml
Connector version semver check
QA checks
Build connector tar
Build destination-mysql-strict-encrypt docker image for platform linux/x86_64
Build airbyte/normalization-mysql:dev
./gradlew :airbyte-integrations:connectors:destination-mysql-strict-encrypt:integrationTest

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-mysql-strict-encrypt test

@octavia-squidington-iii
Copy link
Collaborator

destination-exasol test report (commit ca28112095) - ❌

⏲️ Total pipeline duration: 07mn51s

Step Result
Validate airbyte-integrations/connectors/destination-exasol/metadata.yaml
Connector version semver check
QA checks
Build connector tar
Build destination-exasol docker image for platform linux/x86_64
./gradlew :airbyte-integrations:connectors:destination-exasol:integrationTest

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-exasol test

@octavia-squidington-iii
Copy link
Collaborator

destination-redshift test report (commit ca28112095) - ✅

⏲️ Total pipeline duration: 28mn28s

Step Result
Validate airbyte-integrations/connectors/destination-redshift/metadata.yaml
Connector version semver check
QA checks
Build connector tar
Build destination-redshift docker image for platform linux/x86_64
Build airbyte/normalization-redshift:dev
./gradlew :airbyte-integrations:connectors:destination-redshift:integrationTest

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-redshift test

@octavia-squidington-iii
Copy link
Collaborator

destination-mssql-strict-encrypt test report (commit ca28112095) - ❌

⏲️ Total pipeline duration: 24mn51s

Step Result
Validate airbyte-integrations/connectors/destination-mssql-strict-encrypt/metadata.yaml
Connector version semver check
QA checks
Build connector tar
Build destination-mssql-strict-encrypt docker image for platform linux/x86_64
Build airbyte/normalization-mssql:dev
./gradlew :airbyte-integrations:connectors:destination-mssql-strict-encrypt:integrationTest

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-mssql-strict-encrypt test

@pedroslopez pedroslopez disabled auto-merge July 19, 2023 04:33
@pedroslopez pedroslopez changed the title workaround for building base-normalization fix pyyaml/cython dependency issue when building base-normalization Jul 19, 2023
@pedroslopez
Copy link
Contributor Author

Connectors Base build failure was due to a failing test on airbyte-cdk that has been fixed in master

@pedroslopez
Copy link
Contributor Author

/approve-and-merge reason="fix build"

@octavia-approvington
Copy link
Contributor

Send it!
rocket socks

@octavia-approvington octavia-approvington merged commit e10f768 into master Jul 19, 2023
@octavia-approvington octavia-approvington deleted the pedroslopez/fix-base-normalization branch July 19, 2023 04:37
@octavia-squidington-iii
Copy link
Collaborator

destination-rockset test report (commit 08ad92b9eb) - ❌

⏲️ Total pipeline duration: 10mn53s

Step Result
Validate airbyte-integrations/connectors/destination-rockset/metadata.yaml
Connector version semver check
QA checks
Build connector tar
Build destination-rockset docker image for platform linux/x86_64
./gradlew :airbyte-integrations:connectors:destination-rockset:integrationTest

🔗 View the logs here

Please note that tests are only run on PR ready for review. Please set your PR to draft mode to not flood the CI engine and upstream service on following commits.
You can run the same pipeline locally on this branch with the airbyte-ci tool with the following command

airbyte-ci connectors --name=destination-rockset test

efimmatytsin pushed a commit to scentbird/airbyte that referenced this pull request Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants