Skip to content

cloud-availability-updater: implement git interactions #21976

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Jan 27, 2023

What

Starts the work on https://github.com/airbytehq/airbyte-cloud/issues/4213
We want to automatically open PRs when a connectors becomes eligible to cloud.
This PR focuses on git interaction logics. PR creation and actual call to the new deploy_new_connector_to_cloud_repo will be done in future PRs.

Prior work:

Future works:

How

Implement the logic to:

  • add a connector to the Airbyte-cloud repo definitions mask
  • locally regenerate the cloud catalog
  • commit / push in a new branch on airbyte-cloud repo

Recommended reading order

def clone_airbyte_cloud_repo(local_repo_path: Path) -> git.Repo:

def test_get_definitions_mask_path(checkout_master, dummy_repo_path: Path):

🚨 User Impact 🚨

None, the code implemented is not actually called at the moment.

@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 14:28 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 14:28 — with GitHub Actions Inactive
@alafanechere alafanechere force-pushed the augustin/cloud-availability-updater-git-interactions branch from e711f2f to 18abf99 Compare January 27, 2023 14:32
@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 14:34 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 14:34 — with GitHub Actions Inactive
@github-actions
Copy link
Contributor

Airbyte Code Coverage

There is no coverage information present for the Files changed

Total Project Coverage 24%

@alafanechere alafanechere marked this pull request as ready for review January 27, 2023 14:45
@alafanechere alafanechere requested a review from a team January 27, 2023 14:46
Copy link
Contributor

@evantahler evantahler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love code that writes code!
Are there any special permissions you will need to run this on GH actions?

logging.warning(f"{connector.connector_name}'s definition id is already in {definitions_mask_path}.")
return None

to_append = f"""# {connector.connector_name} (from cloud availability updater)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think I understand how this will have all the YAML properties form the OSS repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only need the connector definition id to add it to the mask.
But we usually comment the name of the connector . e.g

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it! We should add a note in the PR that this creates then about "...and if you you want to modify the connector (e.g. resourceRequirements) do so before merging this PR"

documentation_is_available=True
)
cloud_availability_updater.deploy_new_connector_to_cloud_repo(repo_path, airbyte_cloud_repo, connector)
new_branch_name = f"cloud-availability-updater/deploy-{connector.connector_technical_name}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 connector names shouldn't have any spaces or special chars

@octavia-squidington-iv octavia-squidington-iv added the area/frontend Related to the Airbyte webapp label Jan 27, 2023
@alafanechere alafanechere force-pushed the augustin/cloud-availability-updater-git-interactions branch from 710dcbe to 18abf99 Compare January 27, 2023 17:24
@octavia-squidington-iv octavia-squidington-iv removed the area/frontend Related to the Airbyte webapp label Jan 27, 2023
@alafanechere alafanechere enabled auto-merge (squash) January 27, 2023 17:26
@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 17:26 — with GitHub Actions Inactive
@alafanechere alafanechere temporarily deployed to more-secrets January 27, 2023 17:27 — with GitHub Actions Inactive
@alafanechere
Copy link
Contributor Author

Are there any special permissions you will need to run this on GH actions?
I expect the Octavia PAT to have these super permissions. It's used in this workflow that commits to master...
If not I'll ask @supertopher 😄

@alafanechere alafanechere merged commit 908a93f into master Jan 27, 2023
@alafanechere alafanechere deleted the augustin/cloud-availability-updater-git-interactions branch January 27, 2023 18:00
@supertopher
Copy link
Contributor

@alafanechere as long as you can commit code you can access secrets which is the only real barrier to GH actions.

they keys mostly have the same permissions, but there are some edge cases

I think most keys are basically users. there are a couple that own the org. If you see a permissions error I think the error message is pretty clear

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants