Skip to content

SAT: test strictness level migration tooling (create issues script) #19020

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 7, 2022

Conversation

alafanechere
Copy link
Contributor

@alafanechere alafanechere commented Nov 7, 2022

What

Addition of scripts to create a GitHub issue for each GA connector we want to migrate to high test strictness level.

This is step 4 of our execution plan.

How

Please check the README.md file:

Example of an issue created with this script

Output of the create_issues.py script in --dry-run:

INFO:root:[DRY RUN]: gh issue create --title Source AlloyDB for PostgreSQL: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpz1ms5j5r --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Amazon Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp9moz2eaf --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Amplitude: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpxq2_o62_ --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Bing Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpi0ok1vip --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Chargebee: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmps_7cwzbs --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
...

@airbytehq airbytehq deleted a comment from github-actions bot Nov 7, 2022
@alafanechere alafanechere marked this pull request as ready for review November 7, 2022 11:05
@alafanechere alafanechere requested a review from a team November 7, 2022 11:06
Copy link
Contributor

@pedroslopez pedroslopez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet! Could you post the dryrun output or just how many issues we'll create?

* `type/enhancement`
* `test-strictness-level`

Issues are added to the following project: `SAT-high-test-strictness-level`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice- I'm guessing this was easier to do than adding to a Zenhub epic?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes :) I'll add to a Zenhub epic manually afterward, but it's a good way of quickly finding everything created by this script.

from jinja2 import Environment, FileSystemLoader

TEMPLATES_FOLDER = "./templates/"
COMMON_ISSUE_LABELS = ["area/connectors", "team/connectors-python", "type/enhancement", "test-strictness-level"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that we also have connector-specific labels (e.g. connectors/source/youtube-analytics) - we may be able to infer these labels from the docker image name but I don't know if we're always following that pattern, so for these purposes it's probably ok to leave out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I was able to infer the labels quite right, but I'm afraid some of these are not matching perfectly the usual naming rule. If a label does not exist the CLI fails. And there's no feature to list existing labels with the CLI.
So I thought it was good enough not to add the connector-specific label.

…ness_level_migration/README.md

Co-authored-by: Pedro S. Lopez <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Nov 7, 2022

Affected Connector Report

NOTE ⚠️ Changes in this PR affect the following connectors. Make sure to do the following as needed:

  • Run integration tests
  • Bump connector version
  • Add changelog
  • Publish the new version

❌ Sources (63)

Connector Version Changelog Publish
source-airtable 0.1.3
source-amazon-ads 0.1.24
source-amazon-seller-partner 0.2.28
source-amazon-sqs 0.1.0
source-amplitude 0.1.17
source-appsflyer 0.1.0
(doc not found)

(not in seed)
source-asana 0.1.4
source-azure-table 0.1.3
source-braintree 0.1.3
source-cart 0.2.0
(changelog missing)
source-chargebee 0.1.16
(diff seed version)
source-commercetools 0.1.0
source-confluence 0.1.1
source-datadog 0.1.0
source-delighted 0.1.4
source-drift 0.2.5
source-facebook-marketing 0.2.72
source-facebook-pages 0.1.6
source-freshcaller 0.1.0
source-freshsales 0.1.2
source-freshservice 0.1.1
source-github 0.3.7
source-gitlab 0.1.6
source-google-ads 0.2.3
source-google-search-console 0.1.18
source-greenhouse 0.3.0
source-harvest 0.1.11
source-instagram 1.0.0
source-iterable 0.1.21
source-klaviyo 0.1.10
source-lemlist 0.1.1
(changelog missing)

(diff seed version)
source-lever-hiring 0.1.3
source-linnworks 0.1.5
(changelog missing)
source-mailchimp 0.2.15
source-mailgun 0.1.0
(doc not found)
source-monday 0.1.4
source-notion 0.1.10
source-okta 0.1.13
source-onesignal 0.1.2
source-openweather 0.1.6
source-outreach 0.1.2
source-pardot 0.1.0
(doc not found)

(not in seed)
source-paystack 0.1.1
source-pinterest 0.1.8
source-pipedrive 0.1.13
source-plaid 0.3.2
(changelog missing)
source-posthog 0.1.7
source-prestashop 0.2.0
(doc not found)
source-quickbooks-singer 0.1.5
(doc not found)
source-recharge 0.2.4
(changelog missing)
source-retently 0.1.2
source-salesforce 1.0.24
source-salesloft 0.1.3
source-sendgrid 0.2.16
source-sentry 0.1.7
source-strava 0.1.2
source-surveymonkey 0.1.11
source-tplcentral 0.1.0
(not in seed)
source-twilio 0.1.13
source-youtube-analytics 0.1.3
source-zendesk-sunshine 0.1.1
source-zendesk-talk 0.1.5
source-zenloop 0.1.3
  • See "Actionable Items" below for how to resolve warnings and errors.

✅ Destinations (0)

Connector Version Changelog Publish
  • See "Actionable Items" below for how to resolve warnings and errors.

Actionable Items

(click to expand)

Category Status Actionable Item
Version
mismatch
The version of the connector is different from its normal variant. Please bump the version of the connector.

doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.
Changelog
doc not found
The connector does not seem to have a documentation file. This can be normal (e.g. basic connector like source-jdbc is not published or documented). Please double-check to make sure that it is not a bug.

changelog missing
There is no chnagelog for the current version of the connector. If you are the author of the current version, please add a changelog.
Publish
not in seed
The connector is not in the seed file (e.g. source_definitions.yaml), so its publication status cannot be checked. This can be normal (e.g. some connectors are cloud-specific, and only listed in the cloud seed file). Please double-check to make sure that it is not a bug.

diff seed version
The connector exists in the seed file, but the latest version is not listed there. This usually means that the latest version is not published. Please use the /publish command to publish the latest version.

@alafanechere
Copy link
Contributor Author

Thanks for your review Pedro! 41 issues will be created:

INFO:root:[DRY RUN]: gh issue create --title Source AlloyDB for PostgreSQL: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpw5m4ko_4 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Amazon Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpoc9_abxy --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Amplitude: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpj5wmhxup --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Bing Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp82hgvxyj --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Chargebee: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpa1983pvu --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Facebook Marketing: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp16o45x1r --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source File: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp9fyjv3zv --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Freshdesk: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpcq520bx3 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source GitHub: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmplzvhhzue --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Google Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp2z197any --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Google Analytics (Universal Analytics): enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpnw8v79qh --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Google Search Console: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpvqy4nzq8 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Google Sheets: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmptzu7r214 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Greenhouse: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpxm88rgxd --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Harvest: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp_v9_1yk6 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source HubSpot: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpkbiia1sl --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Instagram: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmprufxeub1 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Intercom: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmphg2jfbni --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Iterable: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp97l2kgk1 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Klaviyo: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpvp4bea1l --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source LinkedIn Ads: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp7jscc0xq --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Mailchimp: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpomkvkmts --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Marketo: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpt9f7wpnq --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Mixpanel: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpvilun3t9 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Notion: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpzcytp5sf --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Paypal Transaction: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpd8te_abi --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Pinterest: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpnt_emr_f --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Postgres: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpf8elu00b --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Recharge: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpv6rt8fa2 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source S3: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp_j_caikj --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Salesforce: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpiagp_y6m --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Slack: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmppeal_t52 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Snapchat Marketing: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp6wqevd5g --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Stripe: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpfcrit1yl --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source SurveyMonkey: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp22olzg11 --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source TikTok Marketing: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpiyd8cr1m --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Twilio: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmp06uo7ku_ --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Zendesk Chat: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpo3roi0hw --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Zendesk Support: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpi37tfo5x --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Zendesk Talk: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmplox1lghy --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:[DRY RUN]: gh issue create --title Source Sentry: enable `high` test strictness level in SAT --body-file /var/folders/48/tgyb8s3579x2y9cwl77yk6ch0000gn/T/tmpyk55y55a --project SAT-high-test-strictness-level --label area/connectors --label team/connectors-python --label type/enhancement --label test-strictness-level
INFO:root:Created 41 new issues.

@alafanechere alafanechere merged commit d815fb9 into master Nov 7, 2022
@alafanechere alafanechere deleted the augustin/migrate-GA-connectors-to-strict-mode branch November 7, 2022 16:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants