Skip to content

Stop publishing java connectors on merge to master #28344

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jul 15, 2023

Conversation

pedroslopez
Copy link
Contributor

@pedroslopez pedroslopez commented Jul 15, 2023

What

Changes our "publish on merge" workfow to ignore java connectors

Example run with languages specified for change on faker + postgres (--modified --language=python):

ci_connector_ops.pipelines: Will publish the following connectors: source-faker

https://github.com/airbytehq/airbyte/actions/runs/5559625325/jobs/10155926192#step:5:535

Example run with no languages specified for change on faker + postgres (--modified)

ci_connector_ops.pipelines: Will publish the following connectors: source-faker, source-alloydb, source-alloydb-strict-encrypt, source-postgres, source-postgres-strict-encrypt

https://github.com/airbytehq/airbyte/actions/runs/5559622521/jobs/10155920377#step:5:532

How

  • Fix up the connectors cmd so filters can be applied after the --modified arg
  • Update the workflow to specify python and low-code languages so java is excluded.

@octavia-squidington-iii octavia-squidington-iii added the area/connectors Connector related issues label Jul 15, 2023
@github-actions
Copy link
Contributor

Before Merging a Connector Pull Request

Wow! What a great pull request you have here! 🎉

To merge this PR, ensure the following has been done/considered for each connector added or updated:

  • PR name follows PR naming conventions
  • Breaking changes are considered. If a Breaking Change is being introduced, ensure an Airbyte engineer has created a Breaking Change Plan and you've followed all steps in the Breaking Changes Checklist
  • Connector version has been incremented in the Dockerfile and metadata.yaml according to our Semantic Versioning for Connectors guidelines
  • Secrets in the connector's spec are annotated with airbyte_secret
  • All documentation files are up to date. (README.md, bootstrap.md, docs.md, etc...)
  • Changelog updated in docs/integrations/<source or destination>/<name>.md with an entry for the new version. See changelog example
  • The connector tests are passing in CI
  • You've updated the connector's metadata.yaml file (new!)
  • If set, you've ensured the icon is present in the platform-internal repo. (Docs)

If the checklist is complete, but the CI check is failing,

  1. Check for hidden checklists in your PR description

  2. Toggle the github label checklist-action-run on/off to re-run the checklist CI.

@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label Jul 15, 2023
Comment on lines +106 to +107
if modified:
selected_connectors_and_files = modified_connectors_and_files
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved this to be before the other filters so they can be applied together

Comment on lines -315 to -317
if ctx.obj["modified"]:
selected_connectors_and_files = get_modified_connectors(get_modified_metadata_files(ctx.obj["modified_files"]))
selected_connectors_names = [connector.technical_name for connector in selected_connectors_and_files.keys()]
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modified connectors were already being filtered out in the previous step, so the selected_connectors_and_files ctx already does this.

This was overwriting the other filters in the publish step

@@ -39,7 +39,7 @@ jobs:
metadata_service_gcs_credentials: ${{ secrets.METADATA_SERVICE_PROD_GCS_CREDENTIALS }}
slack_webhook_url: ${{ secrets.PUBLISH_ON_MERGE_SLACK_WEBHOOK }}
spec_cache_gcs_credentials: ${{ secrets.SPEC_CACHE_SERVICE_ACCOUNT_KEY_PUBLISH }}
subcommand: "connectors --concurrency=1 --execute-timeout=3600 --modified publish --main-release"
subcommand: "connectors --concurrency=1 --execute-timeout=3600 --modified --language=python --language=low-code publish --main-release"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Skip java connectors by specifying we only want python and low-code connectors

@github-actions
Copy link
Contributor

Coverage report for source-postgres

There is no coverage information present for the Files changed

Total Project Coverage 70.05% 🍏

@pedroslopez pedroslopez changed the title Pedroslopez/no java publish Stop publishing java connectors on merge to master Jul 15, 2023
@pedroslopez pedroslopez marked this pull request as ready for review July 15, 2023 01:25
@pedroslopez pedroslopez requested review from bnchrch and a team July 15, 2023 01:25
Copy link
Contributor

@bnchrch bnchrch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great use of the language flag!

@pedroslopez pedroslopez merged commit 7befa93 into master Jul 15, 2023
@pedroslopez pedroslopez deleted the pedroslopez/no-java-publish branch July 15, 2023 01:53
efimmatytsin pushed a commit to scentbird/airbyte that referenced this pull request Jul 27, 2023
* process language after modified flag

* modify some connectors

* empty commit

* another bump

* modified connectors are already filtered

* change some connectors

* change some connectors again

* revert to master version

* nit

* filter for python/lowcode connectors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants