🎉Categorized Config Errors Accurately for Google Analytics 4 (GA4) and Google Ads #25987

lazebnyi · 2023-05-11T11:56:52Z

What

Google Analytics 4 (GA4)

The errors encountered with GAv4 should be classified as configuration errors rather than system errors.

Error 403: Client Error - Forbidden for URL: https://analyticsdata.googleapis.com/v1beta/properties/164523383/metadata
Error 400: Client Error - Bad Request for URL: https://analyticsdata.googleapis.com/v1beta/properties/UA-164523383-1/metadata
Error 400: Client Error - Bad Request for URL: https://analyticsdata.googleapis.com/v1beta/properties/a164523383w230249301p216691667/metadata

The error message should be revised to: "Access was denied to the entered Property ID. Please verify your access to the Property ID or refer to Google Analytics' documentation to find your Property ID."

Further enhancements to the specification:
The tooltip should display an example value.
The Property ID field should only accept numeric values and disallow entries such as "UA-165496".

Google Ads

The classification of these errors should be changed from system errors to configuration errors:

Error: Incorrect GAQL query statement: '"SELECT campaign.id, campaign.name FROM campaign"'
Error: Incorrect GAQL query statement: 'customer_query_2'
Error: Incorrect GAQL query statement: 'keyword_stats'

How

With try..except catch error and raise AirbyteTracedException with config_error failure type.

github-actions · 2023-05-11T11:57:12Z

lazebnyi · 2023-05-11T11:59:24Z

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4947619403
❌ connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4947619403
🐛 https://gradle.com/s/3hkktlefeqdii

Build Failed

Test summary info:

Could not find result summary

lazebnyi · 2023-05-11T11:59:47Z

/test connector=connectors/source-google-ads

🕑 connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/4947622532
✅ connectors/source-google-ads https://github.com/airbytehq/airbyte/actions/runs/4947622532
Python tests coverage:

Name                                       Stmts   Miss  Cover
--------------------------------------------------------------
source_google_ads/models.py                   18      0   100%
source_google_ads/__init__.py                  2      0   100%
source_google_ads/utils.py                    61      1    98%
source_google_ads/streams.py                 195      6    97%
source_google_ads/source.py                   95      7    93%
source_google_ads/custom_query_stream.py      56      5    91%
source_google_ads/google_ads.py               73     12    84%
--------------------------------------------------------------
TOTAL                                        500     31    94%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:100: The previous and actual specifications are identical.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
================== 39 passed, 2 skipped in 600.65s (0:10:00) ===================

bazarnov

Looks good, but the question about using the list instead of an or statement.

For Google-Analytics-data-api, please check the flakeCheck errors https://github.com/airbytehq/airbyte/actions/runs/4947619403/jobs/8847290268#step:11:717

bazarnov · 2023-05-11T12:53:14Z

...tions/connectors/source-google-analytics-data-api/source_google_analytics_data_api/source.py

+            stream = GoogleAnalyticsDataApiMetadataStream(config=config, authenticator=config["authenticator"])
+            metadata = next(stream.read_records(sync_mode=SyncMode.full_refresh), None)
+        except HTTPError as e:
+            if e.response.status_code == HTTPStatus.BAD_REQUEST or e.response.status_code == HTTPStatus.FORBIDDEN:


Why not to use a list of errors here, instead, like:

if e.response.status_code in [HTTPStatus.BAD_REQUEST, HTTPStatus.FORBIDDEN]: ....

…google-analytics-v4-config-error

lazebnyi · 2023-05-12T16:52:17Z

/publish connector=connectors/source-google-ads

🕑 Publishing the following connectors:
connectors/source-google-ads
https://github.com/airbytehq/airbyte/actions/runs/4961191309

Connector	Version	Did it publish?	Were definitions generated?
connectors/source-google-ads	0.2.17	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

lazebnyi · 2023-05-12T18:21:57Z

/test connector=connectors/source-google-analytics-data-api

🕑 connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4961883441
✅ connectors/source-google-analytics-data-api https://github.com/airbytehq/airbyte/actions/runs/4961883441
Python tests coverage:

Name                                                Stmts   Miss  Cover
-----------------------------------------------------------------------
source_google_analytics_data_api/__init__.py            2      0   100%
source_google_analytics_data_api/authenticator.py      44      2    95%
source_google_analytics_data_api/utils.py              26      2    92%
source_google_analytics_data_api/source.py            237     28    88%
source_google_analytics_data_api/api_quota.py          94     12    87%
-----------------------------------------------------------------------
TOTAL                                                 403     44    89%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:109: Backward compatibility tests are disabled for version 0.2.1.
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/connector_acceptance_test/tests/test_core.py:578: The previous and actual discovered catalogs are identical.
============= 38 passed, 2 skipped, 1 warning in 431.86s (0:07:11) =============

lazebnyi · 2023-05-12T18:44:38Z

/publish connector=connectors/source-google-analytics-data-api

🕑 Publishing the following connectors:
connectors/source-google-analytics-data-api
https://github.com/airbytehq/airbyte/actions/runs/4962045164

Connector	Version	Did it publish?	Were definitions generated?
connectors/source-google-analytics-data-api	0.2.2	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

…d Google Ads (airbytehq#25987) * Categorized Config Errors Accurately * Update PR number * Update error list for GAv4 * Updated version * Updated formating * auto-bump connector version * Skip spec backward compatibility * auto-bump connector version --------- Co-authored-by: Octavia Squidington III <[email protected]>

Categorized Config Errors Accurately

0f4a9cb

octavia-squidington-iii added area/connectors Connector related issues area/documentation Improvements or additions to documentation connectors/source/google-ads connectors/source/google-analytics-data-api labels May 11, 2023

Update PR number

b2a8245

lazebnyi requested review from grubberr, davydov-d and bazarnov May 11, 2023 12:00

bazarnov approved these changes May 11, 2023

View reviewed changes

lazebnyi and others added 4 commits May 12, 2023 18:24

Update error list for GAv4

56b6144

Merge branch 'master' into lazebnyi/categorize-source-google-ads-and-…

3a8b047

…google-analytics-v4-config-error

Updated version

b807861

Updated formating

38fffbd

octavia-squidington-iii and others added 3 commits May 12, 2023 17:15

auto-bump connector version

dcd4f7c

Skip spec backward compatibility

9fb1a43

Merged master to branch

4dc9e55

auto-bump connector version

2f2b9b0

lazebnyi merged commit 1060415 into master May 12, 2023

lazebnyi deleted the lazebnyi/categorize-source-google-ads-and-google-analytics-v4-config-error branch May 12, 2023 19:39

This was linked to issues May 12, 2023

Categorize Google Ads config erros #25689

Closed

Categorize Google Analytics 4 (GA4) config errors #25692

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🎉Categorized Config Errors Accurately for Google Analytics 4 (GA4) and Google Ads #25987

🎉Categorized Config Errors Accurately for Google Analytics 4 (GA4) and Google Ads #25987

lazebnyi commented May 11, 2023 •

edited

Loading

github-actions bot commented May 11, 2023 •

edited by lazebnyi

Loading

lazebnyi commented May 11, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 11, 2023 •

edited by github-actions bot

Loading

bazarnov left a comment •

edited

Loading

bazarnov May 11, 2023

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading

🎉Categorized Config Errors Accurately for Google Analytics 4 (GA4) and Google Ads #25987

🎉Categorized Config Errors Accurately for Google Analytics 4 (GA4) and Google Ads #25987

Conversation

lazebnyi commented May 11, 2023 • edited Loading

What

Google Analytics 4 (GA4)

Google Ads

How

github-actions bot commented May 11, 2023 • edited by lazebnyi Loading

Before Merging a Connector Pull Request

lazebnyi commented May 11, 2023 • edited by github-actions bot Loading

Build Failed

lazebnyi commented May 11, 2023 • edited by github-actions bot Loading

Build Passed

bazarnov left a comment • edited Loading

Choose a reason for hiding this comment

bazarnov May 11, 2023

Choose a reason for hiding this comment

lazebnyi commented May 12, 2023 • edited by github-actions bot Loading

lazebnyi commented May 12, 2023 • edited by github-actions bot Loading

Build Passed

lazebnyi commented May 12, 2023 • edited by github-actions bot Loading

lazebnyi commented May 11, 2023 •

edited

Loading

github-actions bot commented May 11, 2023 •

edited by lazebnyi

Loading

lazebnyi commented May 11, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 11, 2023 •

edited by github-actions bot

Loading

bazarnov left a comment •

edited

Loading

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading

lazebnyi commented May 12, 2023 •

edited by github-actions bot

Loading