connectors-qa: introduce run_on_released_connectors flag #36818
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
This stack of pull requests is managed by Graphite. Learn more about stacking. Join @alafanechere and the rest of your teammates on |
alafanechere
force-pushed
the
augustin/04-04-connectors-qa_Introduce_the_Check.run_on_released_connectors_flag
branch
from
1eb23c5 to
8deb9a3
Compare
| @@ -555,6 +556,20 @@ def normalization_tag(self) -> Optional[str]: | |||
| def is_using_poetry(self) -> bool: | |||
| return Path(self.code_directory / "pyproject.toml").exists() | |||
|
|
|||
| @property | |||
| def is_released(self) -> bool: | |||
There was a problem hiding this comment.
Nit: can you add a comment line here? Pretty obvious, sure, but why use OSS catalog for example?
There was a problem hiding this comment.
Should we for example insist on running https checks when we're pushing a connector from being available on OSS only to being available in cloud? Should those be two separate release levels?
There was a problem hiding this comment.
but why use OSS catalog for example?
The cloud registry is a subset of the oss registry. Some connector are not on cloud but I still want to know if they are released.
| registry = download_catalog(OSS_CATALOG_URL) | ||
| for connector in registry[f"{self.connector_type}s"]: | ||
| if ( | ||
| connector[f"{self.connector_type}DefinitionId"] == metadata["definitionId"] |
There was a problem hiding this comment.
Wait a second, does each version of a connector have a different definition ID? Because we want to run all checks on each new version in case new version screws things up.
There was a problem hiding this comment.
Definitions IDs are static and are connector's primary key. The logic here is:
If the definition ID and the dockerImage version are found in the registry it means the connector was published.
| @property | ||
| def is_released(self) -> bool: | ||
| metadata = self.metadata | ||
| in_registry = False |
There was a problem hiding this comment.
| in_registry = False |
| connector[f"{self.connector_type}DefinitionId"] == metadata["definitionId"] | ||
| and connector["dockerImageTag"] == metadata["dockerImageTag"] | ||
| ): | ||
| in_registry = True |
There was a problem hiding this comment.
| in_registry = True | |
| return True |
| and connector["dockerImageTag"] == metadata["dockerImageTag"] | ||
| ): | ||
| in_registry = True | ||
| break |
There was a problem hiding this comment.
| break | |
| return False |
| ): | ||
| in_registry = True | ||
| break | ||
| return in_registry |
There was a problem hiding this comment.
| return in_registry |
alafanechere
force-pushed
the
augustin/04-04-connectors-qa_Introduce_the_Check.run_on_released_connectors_flag
branch
from
8deb9a3 to
4da8624
Compare
alafanechere
deleted the
augustin/04-04-connectors-qa_Introduce_the_Check.run_on_released_connectors_flag
branch
We have some check that are not meant to run on nightly builds, for released connectors.
This PR introduce a flag that can be set at the
Checklevel to determine if the check should run on a released connector or be skipped.The released state of the connector is determined by the presence of the current version in metadata in the OSS registry.