metadata-service[lib]: introduce a `rolloutConfiguration` field to the metadata model #44504

alafanechere · 2024-08-21T12:59:29Z

What

Closes https://github.com/airbytehq/airbyte-internal-issues/issues/9249

How

Add a RolloutConfiguration.yaml model
Reference it in the connector metadata model under the releases field.
Introduce a isReleaseCandidate field in under releases
Re-generated pydantic models

vercel · 2024-08-21T12:59:33Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
airbyte-docs	⬜️ Ignored (Inspect)	Visit Preview		Aug 22, 2024 1:17pm

alafanechere · 2024-08-21T12:59:47Z

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @alafanechere and the rest of your teammates on Graphite

alafanechere · 2024-08-21T13:23:03Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/Rollout.yaml

+type: object
+additionalProperties: false
+properties:
+  initialPercentage:


@clnoll I was wondering if we plan to version control the rollout progress. Shall we introduce a currentPercentage which would be updated via automated PRs by the orchestrator?

@alafanechere I wasn't planning to. I'd prefer to keep the flow of information in one direction, and encourage people to use information from the RC table as the point of truth.

clnoll · 2024-08-21T13:53:44Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/Rollout.yaml

+    minimum: 0
+    maximum: 100
+    default: 100
+    description: The maximum percentage of users that should receive the new version.


Suggested change

description: The maximum percentage of users that should receive the new version.

description: The percentage of users who should receive the release candidate during the test phase before full rollout.

clnoll · 2024-08-21T13:55:26Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/validators/metadata_validator.py

+def validate_no_rollout_on_major_version(
+    metadata_definition: ConnectorMetadataDefinitionV0, _validator_opts: ValidatorOptions
+) -> ValidationResult:
+    """Ensure that if the major version is incremented, there is a breaking change entry for that version."""


Suggested change

"""Ensure that if the major version is incremented, there is a breaking change entry for that version."""

"""Ensure that gradual rollout is not performed on a major version bump."""

clnoll · 2024-08-21T13:57:30Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/validators/metadata_validator.py

+        return True, None
+
+    rollout = get(metadata_definition_dict, "data.rollout")
+    if rollout:


I don't think we should remove the rollout property on a major version bump; we should be able to just keep it there. Instead we can just do the normal publishing flow (not publish a release candidate).

clnoll · 2024-08-21T15:31:13Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/Rollout.yaml

+    type: integer
+    minimum: 0
+    maximum: 100
+    default: 100


WDYT about defaulting this to 50 instead? That would mean that up to half of the connectors will get the RC before it gets fully rolled out.

clnoll · 2024-08-21T16:37:59Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/ConnectorReleases.yaml

@@ -8,6 +8,12 @@ additionalProperties: false
 required:
  - breakingChanges
 properties:
+  isReleaseCandidate:


Suggested change

isReleaseCandidate:

isEligibleReleaseCandidate:

clnoll

LGTM, just a few wording suggestions.

clnoll · 2024-08-21T16:38:22Z

airbyte-ci/connectors/metadata_service/lib/metadata_service/models/src/ConnectorReleases.yaml

@@ -8,6 +8,12 @@ additionalProperties: false
 required:
  - breakingChanges
 properties:
+  isReleaseCandidate:
+    description: Whether the release is a release candidate.


Suggested change

description: Whether the release is a release candidate.

description: Whether the release is eligible to be a release candidate.

clnoll · 2024-08-21T16:38:57Z

...yte-ci/connectors/metadata_service/lib/metadata_service/models/src/RolloutConfiguration.yaml

+    minimum: 0
+    maximum: 100
+    default: 50
+    description: The maximum percentage of users that should receive the new version.


Suggested change

description: The maximum percentage of users that should receive the new version.

description: The percentage of users that should receive the release candidate during the test phase before full rollout.

alafanechere · 2024-08-22T16:07:28Z

Merge activity

Aug 22, 12:07 PM EDT: @alafanechere started a stack merge that includes this pull request via Graphite.
Aug 22, 12:07 PM EDT: @alafanechere merged this pull request with Graphite.

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from 9454b01 to 230d6ea Compare August 21, 2024 13:05

alafanechere changed the title ~~metadata-service[lib]: introduce the Rollout field to metadata model~~ metadata-service[lib]: introduce a rollout field to the metadata model Aug 21, 2024

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from 230d6ea to adcc8ce Compare August 21, 2024 13:19

alafanechere marked this pull request as ready for review August 21, 2024 13:21

alafanechere requested review from a team and clnoll August 21, 2024 13:21

alafanechere commented Aug 21, 2024

View reviewed changes

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from adcc8ce to e759b7f Compare August 21, 2024 13:28

clnoll reviewed Aug 21, 2024

View reviewed changes

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from e759b7f to bbdcb0a Compare August 21, 2024 16:05

alafanechere mentioned this pull request Aug 21, 2024

metadata-service[lib]: change metadata upload logic for release candidates #44509

Merged

alafanechere changed the title ~~metadata-service[lib]: introduce a rollout field to the metadata model~~ metadata-service[lib]: introduce a rolloutConfiguration field to the metadata model Aug 21, 2024

alafanechere requested a review from clnoll August 21, 2024 16:11

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch 3 times, most recently from 3ccc18d to c678426 Compare August 21, 2024 16:24

clnoll reviewed Aug 21, 2024

View reviewed changes

clnoll approved these changes Aug 21, 2024

View reviewed changes

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from c678426 to 46977fc Compare August 22, 2024 12:25

metadata-service[lib]: introduce the Rollout field to metadata model

c088c5b

alafanechere force-pushed the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch from 46977fc to c088c5b Compare August 22, 2024 13:17

alafanechere mentioned this pull request Aug 22, 2024

airbyte-ci: do not push latest tag for release candidates #44551

Merged

alafanechere merged commit f58652b into master Aug 22, 2024
30 checks passed

alafanechere deleted the augustin/08-21-metadata-service_lib_introduce_the_Rollout_field_to_metadata_model branch August 22, 2024 16:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

metadata-service[lib]: introduce a `rolloutConfiguration` field to the metadata model #44504

metadata-service[lib]: introduce a `rolloutConfiguration` field to the metadata model #44504

alafanechere commented Aug 21, 2024 •

edited

Loading

vercel bot commented Aug 21, 2024 •

edited

Loading

alafanechere commented Aug 21, 2024 •

edited

Loading

alafanechere Aug 21, 2024

clnoll Aug 21, 2024

alafanechere Aug 21, 2024

clnoll Aug 21, 2024

clnoll Aug 21, 2024

clnoll Aug 21, 2024

clnoll Aug 21, 2024

clnoll Aug 21, 2024

clnoll left a comment

clnoll Aug 21, 2024

clnoll Aug 21, 2024

alafanechere commented Aug 22, 2024 •

edited

Loading

	description: The maximum percentage of users that should receive the new version.
	description: The percentage of users who should receive the release candidate during the test phase before full rollout.

	"""Ensure that if the major version is incremented, there is a breaking change entry for that version."""
	"""Ensure that gradual rollout is not performed on a major version bump."""

	description: Whether the release is a release candidate.
	description: Whether the release is eligible to be a release candidate.

	description: The maximum percentage of users that should receive the new version.
	description: The percentage of users that should receive the release candidate during the test phase before full rollout.

metadata-service[lib]: introduce a rolloutConfiguration field to the metadata model #44504

metadata-service[lib]: introduce a rolloutConfiguration field to the metadata model #44504

Conversation

alafanechere commented Aug 21, 2024 • edited Loading

What

How

vercel bot commented Aug 21, 2024 • edited Loading

alafanechere commented Aug 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clnoll left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alafanechere commented Aug 22, 2024 • edited Loading

Merge activity

metadata-service[lib]: introduce a `rolloutConfiguration` field to the metadata model #44504

metadata-service[lib]: introduce a `rolloutConfiguration` field to the metadata model #44504

alafanechere commented Aug 21, 2024 •

edited

Loading

vercel bot commented Aug 21, 2024 •

edited

Loading

alafanechere commented Aug 21, 2024 •

edited

Loading

alafanechere commented Aug 22, 2024 •

edited

Loading