Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

airbyte-ci format: make most inputs optional in run-airbyte-ci #37683

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Apr 30, 2024

What

Continue to make format automatically run on fork but clean up the work introduced in #37356 to use the consistent approach in installing and running airbyte-ci via the run-airbyte-ci re-usable action.

How

  • Remove the specific "format on fork" job of the format_check.yml workflow
  • Create a format job in the community_ci.yml workflow. Make it run automatically in a new commmunity-ci-auto environmnet.
  • Make most run-airbyte-ci inputs optional to avoid requiring secret passing in format.
  • Clean up / fix typos in existing workflows

Copy link

vercel bot commented Apr 30, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Apr 30, 2024 2:26pm

Copy link
Contributor Author

alafanechere commented Apr 30, 2024

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @alafanechere and the rest of your teammates on Graphite Graphite

@alafanechere alafanechere force-pushed the augustin/04-30-airbyte-ci_format_make_most_inputs_optional_in_run-airbyte-ci branch 11 times, most recently from ece3999 to 755156d Compare April 30, 2024 10:09
@@ -36,8 +36,10 @@ runs:
if [[ "${{ github.ref }}" != "refs/heads/master" ]] && [[ "${{ steps.changes.outputs.pipelines_any_changed }}" == "true" ]]; then
echo "Making changes to Airbyte CI on a non-master branch. Airbyte-CI will be installed from source."
echo "install-mode=source" >> $GITHUB_OUTPUT
echo "SENTRY_ENVIRONMENT=dev" >> $GITHUB_ENV
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved this here because I figure it was not properly declared in run-airbyte-ci. It make more sense to declare this env var here:
When we're developing on airbyte-ci we don't want to create noise in sentry so we set the sentry env to dev.
When we're using airbyte-ci binary (not developing on it) we want to set the sentry env to prod.

@@ -93,15 +93,14 @@ runs:
- name: Get start timestamp
id: get-start-timestamp
shell: bash
run: echo "name=start-timestamp=$(date +%s)" >> $GITHUB_OUTPUT
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

name= was a typo...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol, how did this even work then?

- name: Docker login
id: docker-login
uses: docker/login-action@v3
if: ${{ inputs.docker_hub_username != '' && inputs.docker_hub_password != '' }}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do no perform docker login if no docker creds were passed by the calling workflow

@alafanechere alafanechere force-pushed the augustin/04-30-airbyte-ci_format_make_most_inputs_optional_in_run-airbyte-ci branch from 755156d to 698ff69 Compare April 30, 2024 12:21
@alafanechere alafanechere marked this pull request as ready for review April 30, 2024 12:23
@alafanechere alafanechere requested a review from a team April 30, 2024 12:23
@alafanechere alafanechere force-pushed the augustin/04-30-airbyte-ci_format_make_most_inputs_optional_in_run-airbyte-ci branch from 698ff69 to 5a55192 Compare April 30, 2024 14:26
Copy link
Contributor

@natikgadzhi natikgadzhi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is already good — and I agree with @aaronsteers, perhaps we should just keep one implementation of format_check that runs on both our PRs and PRs from forks? If there is nothing priviledged, no extra env is needed?

@@ -93,15 +93,14 @@ runs:
- name: Get start timestamp
id: get-start-timestamp
shell: bash
run: echo "name=start-timestamp=$(date +%s)" >> $GITHUB_OUTPUT
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol, how did this even work then?

SLACK_WEBHOOK: ${{ inputs.slack_webhook_url }}
SPEC_CACHE_BUCKET_NAME: ${{ inputs.spec_cache_bucket_name }}
SPEC_CACHE_GCS_CREDENTIALS: ${{ inputs.spec_cache_gcs_credentials }}
# give the Dagger Engine more time to push cache data to Dagger Cloud
run: |
airbyte-ci --disable-update-check --disable-dagger-run --is-ci --gha-workflow-run-id=${{ github.run_id }} ${{ inputs.subcommand }} ${{ inputs.options }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL there's disable-dagger-run!

if: always()
uses: jwalton/gh-docker-logs@v2
with:
dest: "./docker_logs"
dest: "./dagger_engine_logs"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is essentially just a rename of the output file, right? Still same / similar logs?

# IMPORTANT: This name must match the require check name on the branch protection settings
name: "Check for formatting errors"
if: github.event.pull_request.head.repo.fork == true
environment: community-ci-auto
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this make it "gated" by a human button press? Do we care?

@alafanechere alafanechere merged commit 9c8fd80 into master Apr 30, 2024
32 checks passed
@alafanechere alafanechere deleted the augustin/04-30-airbyte-ci_format_make_most_inputs_optional_in_run-airbyte-ci branch April 30, 2024 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants