🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping #17873

rach-r · 2022-10-12T09:02:25Z

What

Today, only Search campaign type is fetched in Campaigns stream, while all campaigns are available in campaign reports.
This PR aims to fix this by explicitly calling all campaign types in request parameters.

Closes this issue.

How

By explicitly calling all campaign types in request parameters.
Here is the API documentation mentioning how it should be retrieved.

Pre-merge Checklist

Updating a connector

Community member or Airbyter

Grant edit access to maintainers (instructions)
Unit & integration tests added and passing. Community members, please provide proof of success locally e.g: screenshot or copy-paste unit, integration, and acceptance test output. To run acceptance tests for a Python connector, follow instructions in the README. For java connectors run ./gradlew :airbyte-integrations:connectors:<name>:integrationTest.
Documentation updated
- Changelog updated in docs/integrations/<source or destination>/<name>.md including changelog. See changelog example
PR name follows PR naming conventions

Tests

Unit

Put your unit tests output here.

Integration

Put your integration tests output here.

Acceptance

Put your acceptance tests output here.

…rams

…are requested

CLAassistant · 2022-10-12T09:02:30Z

All committers have signed the CLA.

alafanechere

Thank you for this initiative!
I'll run the acceptance tests on our sandbox account below.

alafanechere · 2022-10-12T10:07:39Z

airbyte-integrations/connectors/source-bing-ads/source_bing_ads/source.py

@@ -207,6 +207,9 @@ class Campaigns(BingAdsStream):
        " DynamicFeedSetting MaxConversionValueBiddingScheme MultimediaAdsBidAdjustment"
        " TargetImpressionShareBiddingScheme TargetSetting VerifiedTrackingSetting"
    )
+    campaign_types: str = (


I'd prefer this attribute to be a list of string that is later joined with " ".join. This will increase readability and simplify testing.
Feel free to change additional_fields in the same manner.

alafanechere · 2022-10-12T10:14:36Z

/test connector=connectors/source-bing-ads

🕑 connectors/source-bing-ads https://github.com/airbytehq/airbyte/actions/runs/3233852670
✅ connectors/source-bing-ads https://github.com/airbytehq/airbyte/actions/runs/3233852670
Python tests coverage:

Name                          Stmts   Miss  Cover
-------------------------------------------------
source_bing_ads/__init__.py       2      0   100%
source_bing_ads/source.py       260      9    97%
source_bing_ads/cache.py         18      1    94%
source_bing_ads/reports.py      133     17    87%
source_bing_ads/client.py        93     14    85%
-------------------------------------------------
TOTAL                           506     41    92%
	 Name                                                 Stmts   Miss  Cover   Missing
	 ----------------------------------------------------------------------------------
	 source_acceptance_test/base.py                          10      4    60%   15-18
	 source_acceptance_test/config.py                        83      6    93%   78-80, 84-86
	 source_acceptance_test/conftest.py                     164    164     0%   6-282
	 source_acceptance_test/plugin.py                        48     48     0%   6-104
	 source_acceptance_test/tests/test_core.py              329    111    66%   39, 50-58, 63-70, 74-75, 79-80, 164, 202-219, 228-236, 240-245, 251, 284-289, 327-334, 374-376, 379, 439-448, 477-478, 484, 487, 520-530, 543-568, 573-577
	 source_acceptance_test/tests/test_full_refresh.py       52      2    96%   34, 65
	 source_acceptance_test/tests/test_incremental.py       152     26    83%   21-23, 29-31, 36-43, 48-61, 239, 250-258
	 source_acceptance_test/utils/asserts.py                 37      2    95%   57-58
	 source_acceptance_test/utils/common.py                  77     17    78%   15-16, 24-30, 47-54, 64, 67
	 source_acceptance_test/utils/compare.py                 62     23    63%   21-51, 68, 97-99
	 source_acceptance_test/utils/connector_runner.py       112     50    55%   23-26, 32, 36, 39-67, 70-72, 75-77, 80-82, 85-87, 90-92, 95-113, 147-149
	 source_acceptance_test/utils/json_schema_helper.py     105     13    88%   30-31, 38, 41, 65-68, 96, 120, 190-192
	 ----------------------------------------------------------------------------------
	 TOTAL                                                 1358    466    66%

Build Passed

Test summary info:

=========================== short test summary info ============================
SKIPPED [1] ../usr/local/lib/python3.9/site-packages/source_acceptance_test/plugin.py:60: Skipping TestIncremental.test_two_sequential_reads because not found in the config
================== 27 passed, 1 skipped in 638.02s (0:10:38) ===================

alafanechere · 2022-10-12T11:27:30Z

Requesting review from @grubberr / @bazarnov for awareness as this connector is GA.

alafanechere

👍 Tests are passing. @rach-r feel free to make the minor suggestion I shared above.

bazarnov

LGTM, but make sure this comment is processed: https://github.com/airbytehq/airbyte/pull/17873/files#r993262907

bazarnov · 2022-10-12T12:31:17Z

airbyte-integrations/connectors/source-bing-ads/source_bing_ads/source.py

@@ -207,6 +207,9 @@ class Campaigns(BingAdsStream):
        " DynamicFeedSetting MaxConversionValueBiddingScheme MultimediaAdsBidAdjustment"
        " TargetImpressionShareBiddingScheme TargetSetting VerifiedTrackingSetting"
    )
+    campaign_types: str = (


…st for Campaigns class

alafanechere · 2022-10-13T10:29:27Z

/publish connector=connectors/source-bing-ads

🕑 Publishing the following connectors:
connectors/source-bing-ads
https://github.com/airbytehq/airbyte/actions/runs/3241609027

Connector	Did it publish?	Were definitions generated?
connectors/source-bing-ads	✅	✅

if you have connectors that successfully published but failed definition generation, follow step 4 here ▶️

alafanechere · 2022-10-14T13:38:48Z

Thanks @rach-r for the contribution! 🎉 Feel free to open more PR like this one ;-)

…vation * master: (98 commits) 🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping (#17873) Source S3 - fix schema inference (#17991) 🎉 JDBC sources: store cursor record count in db state (#15535) Introduce webhook configs into workspace api and persistence (#17950) ci: upload test results to github for analysis (#17953) Trigger the connectors build if there are worker changes. (#17976) Add additional sync timing information (#17643) Use page_token_option instead of page_token (#17892) capture metrics around json messages size (#17973) 🐛 Correct kube annotations variable as per the docs. (#17972) 🪟 🎉 Add /connector-builder page with embedded YAML editor (#17482) fix `est_num_metrics_emitted_by_reporter` not being emitted (#17929) Update schema dumps (#17960) Remove the bump in the value.yml (#17959) Ensure database initialization in test container (#17697) Remove typo line from incremental reads docs (#17920) DocS: Update authentication.md (#17931) Use MessageMigration for Source Connection Check. (#17656) fixed links (#17949) remove usages of YamlSeedConfigPersistence (#17895) ...

lluisgassovillarejo · 2022-10-17T09:02:23Z

Hi,
We have found this campaign-type definition problem also in the Cloud version.

The problem summarizes that we are making a move from Fivetran to Airbyte Cloud and:

Fivetran has three campaigns: Shopping, Search, and Audience.
On Airbyte, only one appears: Search.

So we find correct but incomplete data.
What can we do to solve this problem?

A greeting and thanks for the help.

…airbytehq#17873)

rach-r added 3 commits October 12, 2022 10:12

fix(source-bing-ads): explicitly add all campaigns type in request pa…

e0a2048

…rams

test(source-bing-ads): update unit tests to check all campaign types …

2e1be57

…are requested

chore(source-bing-ads): bump version to 0.1.16

a19ed4a

github-actions bot added the area/connectors Connector related issues label Oct 12, 2022

octavia-squidington-iv added community connectors/source/bing-ads labels Oct 12, 2022

docs(souce-bing-ads): update changelog

53c74b0

github-actions bot added the area/documentation Improvements or additions to documentation label Oct 12, 2022

alafanechere reviewed Oct 12, 2022

View reviewed changes

Merge branch 'master' into fix-missing-campaigns-bing-ads

46c71c4

alafanechere marked this pull request as ready for review October 12, 2022 10:34

octavia-squidington-iv added the bounty label Oct 12, 2022

alafanechere requested review from grubberr and bazarnov October 12, 2022 11:27

alafanechere approved these changes Oct 12, 2022

View reviewed changes

bazarnov approved these changes Oct 12, 2022

View reviewed changes

refactor(source-bing-ads): additional_fields and campaign_types as li…

7d62993

…st for Campaigns class

bazarnov approved these changes Oct 12, 2022

View reviewed changes

sajarin added bounty-M Maintainer program: claimable medium bounty PR internal and removed bounty bounty-M Maintainer program: claimable medium bounty PR labels Oct 12, 2022

Merge branch 'master' into fix-missing-campaigns-bing-ads

e498138

auto-bump connector version [ci skip]

b50487a

alafanechere merged commit cdb8052 into airbytehq:master Oct 14, 2022

octavia-squidington-iii mentioned this pull request Oct 20, 2022

Bump Airbyte version from 0.40.15 to 0.40.16 #18268

Merged

jhammarstedt pushed a commit to jhammarstedt/airbyte that referenced this pull request Oct 31, 2022

🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping (…

6667a95

…airbytehq#17873)

rach-r mentioned this pull request Mar 7, 2023

Source Bing Ads: Add other CampaignTypes #18303

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping #17873

🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping #17873

Uh oh!

rach-r commented Oct 12, 2022 •

edited

Loading

Uh oh!

CLAassistant commented Oct 12, 2022 •

edited

Loading

Uh oh!

alafanechere left a comment

Uh oh!

alafanechere Oct 12, 2022

Uh oh!

bazarnov Oct 12, 2022

Uh oh!

alafanechere commented Oct 12, 2022 •

edited by github-actions bot

Loading

Uh oh!

alafanechere commented Oct 12, 2022

Uh oh!

alafanechere left a comment

Uh oh!

bazarnov left a comment

Uh oh!

bazarnov Oct 12, 2022

Uh oh!

alafanechere commented Oct 13, 2022 •

edited by github-actions bot

Loading

Uh oh!

alafanechere commented Oct 14, 2022

Uh oh!

lluisgassovillarejo commented Oct 17, 2022

Uh oh!

Uh oh!

🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping #17873

🐛 Source Bing Ads - Fix Campaigns stream misses Audience and Shopping #17873

Uh oh!

Conversation

rach-r commented Oct 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

How

Pre-merge Checklist

Community member or Airbyter

Tests

Uh oh!

CLAassistant commented Oct 12, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alafanechere left a comment

Choose a reason for hiding this comment

Uh oh!

alafanechere Oct 12, 2022

Choose a reason for hiding this comment

Uh oh!

bazarnov Oct 12, 2022

Choose a reason for hiding this comment

Uh oh!

alafanechere commented Oct 12, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Build Passed

Uh oh!

alafanechere commented Oct 12, 2022

Uh oh!

alafanechere left a comment

Choose a reason for hiding this comment

Uh oh!

bazarnov left a comment

Choose a reason for hiding this comment

Uh oh!

bazarnov Oct 12, 2022

Choose a reason for hiding this comment

Uh oh!

alafanechere commented Oct 13, 2022 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alafanechere commented Oct 14, 2022

Uh oh!

lluisgassovillarejo commented Oct 17, 2022

Uh oh!

Uh oh!

rach-r commented Oct 12, 2022 •

edited

Loading

CLAassistant commented Oct 12, 2022 •

edited

Loading

alafanechere commented Oct 12, 2022 •

edited by github-actions bot

Loading

alafanechere commented Oct 13, 2022 •

edited by github-actions bot

Loading