-
Notifications
You must be signed in to change notification settings - Fork 4.6k
SAT: test strictness level migration tooling (create issues script) #19020
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SAT: test strictness level migration tooling (create issues script) #19020
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sweet! Could you post the dryrun output or just how many issues we'll create?
airbyte-integrations/bases/source-acceptance-test/tools/strictness_level_migration/README.md
Outdated
Show resolved
Hide resolved
* `type/enhancement` | ||
* `test-strictness-level` | ||
|
||
Issues are added to the following project: `SAT-high-test-strictness-level` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice- I'm guessing this was easier to do than adding to a Zenhub epic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes :) I'll add to a Zenhub epic manually afterward, but it's a good way of quickly finding everything created by this script.
from jinja2 import Environment, FileSystemLoader | ||
|
||
TEMPLATES_FOLDER = "./templates/" | ||
COMMON_ISSUE_LABELS = ["area/connectors", "team/connectors-python", "type/enhancement", "test-strictness-level"] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that we also have connector-specific labels (e.g. connectors/source/youtube-analytics
) - we may be able to infer these labels from the docker image name but I don't know if we're always following that pattern, so for these purposes it's probably ok to leave out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes I was able to infer the labels quite right, but I'm afraid some of these are not matching perfectly the usual naming rule. If a label does not exist the CLI fails. And there's no feature to list existing labels with the CLI.
So I thought it was good enough not to add the connector-specific label.
…ness_level_migration/README.md Co-authored-by: Pedro S. Lopez <[email protected]>
Affected Connector ReportNOTE
|
Connector | Version | Changelog | Publish |
---|---|---|---|
source-airtable |
0.1.3 |
✅ | ✅ |
source-amazon-ads |
0.1.24 |
✅ | ✅ |
source-amazon-seller-partner |
0.2.28 |
✅ | ✅ |
source-amazon-sqs |
0.1.0 |
✅ | ✅ |
source-amplitude |
0.1.17 |
✅ | ✅ |
source-appsflyer |
0.1.0 |
⚠ (doc not found) |
⚠ (not in seed) |
source-asana |
0.1.4 |
✅ | ✅ |
source-azure-table |
0.1.3 |
✅ | ✅ |
source-braintree |
0.1.3 |
✅ | ✅ |
source-cart |
0.2.0 |
❌ (changelog missing) |
✅ |
source-chargebee |
0.1.16 |
✅ | ❌ (diff seed version) |
source-commercetools |
0.1.0 |
✅ | ✅ |
source-confluence |
0.1.1 |
✅ | ✅ |
source-datadog |
0.1.0 |
✅ | ✅ |
source-delighted |
0.1.4 |
✅ | ✅ |
source-drift |
0.2.5 |
✅ | ✅ |
source-facebook-marketing |
0.2.72 |
✅ | ✅ |
source-facebook-pages |
0.1.6 |
✅ | ✅ |
source-freshcaller |
0.1.0 |
✅ | ✅ |
source-freshsales |
0.1.2 |
✅ | ✅ |
source-freshservice |
0.1.1 |
✅ | ✅ |
source-github |
0.3.7 |
✅ | ✅ |
source-gitlab |
0.1.6 |
✅ | ✅ |
source-google-ads |
0.2.3 |
✅ | ✅ |
source-google-search-console |
0.1.18 |
✅ | ✅ |
source-greenhouse |
0.3.0 |
✅ | ✅ |
source-harvest |
0.1.11 |
✅ | ✅ |
source-instagram |
1.0.0 |
✅ | ✅ |
source-iterable |
0.1.21 |
✅ | ✅ |
source-klaviyo |
0.1.10 |
✅ | ✅ |
source-lemlist |
0.1.1 |
❌ (changelog missing) |
❌ (diff seed version) |
source-lever-hiring |
0.1.3 |
✅ | ✅ |
source-linnworks |
0.1.5 |
❌ (changelog missing) |
✅ |
source-mailchimp |
0.2.15 |
✅ | ✅ |
source-mailgun |
0.1.0 |
⚠ (doc not found) |
✅ |
source-monday |
0.1.4 |
✅ | ✅ |
source-notion |
0.1.10 |
✅ | ✅ |
source-okta |
0.1.13 |
✅ | ✅ |
source-onesignal |
0.1.2 |
✅ | ✅ |
source-openweather |
0.1.6 |
✅ | ✅ |
source-outreach |
0.1.2 |
✅ | ✅ |
source-pardot |
0.1.0 |
⚠ (doc not found) |
⚠ (not in seed) |
source-paystack |
0.1.1 |
✅ | ✅ |
source-pinterest |
0.1.8 |
✅ | ✅ |
source-pipedrive |
0.1.13 |
✅ | ✅ |
source-plaid |
0.3.2 |
❌ (changelog missing) |
✅ |
source-posthog |
0.1.7 |
✅ | ✅ |
source-prestashop |
0.2.0 |
⚠ (doc not found) |
✅ |
source-quickbooks-singer |
0.1.5 |
⚠ (doc not found) |
✅ |
source-recharge |
0.2.4 |
❌ (changelog missing) |
✅ |
source-retently |
0.1.2 |
✅ | ✅ |
source-salesforce |
1.0.24 |
✅ | ✅ |
source-salesloft |
0.1.3 |
✅ | ✅ |
source-sendgrid |
0.2.16 |
✅ | ✅ |
source-sentry |
0.1.7 |
✅ | ✅ |
source-strava |
0.1.2 |
✅ | ✅ |
source-surveymonkey |
0.1.11 |
✅ | ✅ |
source-tplcentral |
0.1.0 |
✅ | ⚠ (not in seed) |
source-twilio |
0.1.13 |
✅ | ✅ |
source-youtube-analytics |
0.1.3 |
✅ | ✅ |
source-zendesk-sunshine |
0.1.1 |
✅ | ✅ |
source-zendesk-talk |
0.1.5 |
✅ | ✅ |
source-zenloop |
0.1.3 |
✅ | ✅ |
- See "Actionable Items" below for how to resolve warnings and errors.
✅ Destinations (0)
Connector | Version | Changelog | Publish |
---|
- See "Actionable Items" below for how to resolve warnings and errors.
Actionable Items
(click to expand)
Category | Status | Actionable Item |
---|---|---|
Version | ❌ mismatch |
The version of the connector is different from its normal variant. Please bump the version of the connector. |
⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
|
Changelog | ⚠ doc not found |
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug. |
❌ changelog missing |
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog. | |
Publish | ⚠ not in seed |
The connector is not in the seed file (e.g. source_definitions.yaml ), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug. |
❌ diff seed version |
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version. |
Thanks for your review Pedro! 41 issues will be created:
|
What
Addition of scripts to create a GitHub issue for each GA connector we want to migrate to
high
test strictness level.This is step 4 of our execution plan.
How
Please check the README.md file:
airbyte/airbyte-integrations/bases/source-acceptance-test/tools/strictness_level_migration/README.md
Line 1 in 389bd86
Example of an issue created with this script
Output of the
create_issues.py
script in--dry-run
: