[EPIC] Destination BigQuery to GA

[alexandr-shegeda]

The main purpose of this issue is to track release progress for the connector.

Acceptance criteria

• Connector should have a separate document that should be placed on the Google Drive
• Collect and list all tech debt-related issues in the section below
• Prepare MLP document for BigQuery Destination #10758
• Update connector documentation according to the comments in the checklist

MLP section

• Support Oauth for bigquery destination #9021
• Destination-Bigquery: Authentication gaps #11089
• [EPIC] Destination-Bigquery: Data type gaps: Numeric, Bignumeric, Bytes, etc #11090
• airbytehq/airbyte-internal-issues#606
• Destination BigQuery: map "airbyte_type": "big_number" to BIGNUMERIC #12658
• Destination BigQuery: timestamps without timezone should be mapped to DATETIME #12660

[misteryeo]

@alexandr-shegeda adding a few comments here:

1. Updated the title so it's explicit that this is an Epic for bringing the connector to GA. Let's follow this for other connectors too.
2. Updated the header from Tech Debt Issues to Outstanding Bugs just to be explicit. For the ones listed here, let's copy this to a separate epic for BigQuery Denormalized. This quarter, we won't have the bandwidth to merge BigQuery Denormalized with BigQuery so we should only be working on BigQuery non-denormalized connector issues if any.
3. MLP section looks good for now - @sherifnada is pulling together a destination template for research similar to what we've been doing for sources to make sure there's no other features we need to add here.

[misteryeo]

@alexandr-shegeda for clarity - BigQuery we're bringing to GA. BigQuery De-normalized we're bringing to Beta first. Let's keep these two efforts separate. Once they're both in GA, we'll allocate time to merge them.

The destination template is now complete as a first draft, can you give that a try to create a research doc for BigQuery and determine if there are more MLP criteria for BigQuery?

[alexandr-shegeda]

@misteryeo moved bigquery-denormalized related tickets into a separate issue airbytehq/airbyte-internal-issues#436

[misteryeo]

@alexandr-shegeda Making sure we're aligned here - OAuth doesn't need to be completed, that's been documented and is sufficient for now. Please go ahead and wrap up the BigQuery checklist spreadsheet. I've added comments there. Thanks!

[terekete]

Will this be released with Oauth ?

[misteryeo]

No, we won't be releasing OAuth with GA @terekete

[terekete]

Is Oauth coming in a future release ? Reasoning I ask, not everyone in industry agrees to use service account keys.

[misteryeo]

@terekete Yep, we'll plan to implement OAuth in a future release but we don't have a set timeframe just yet. I'd added your request for us to keep track and will continue to take requests from other users of BigQuery as well to help us prioritize.

[sherifnada]

@alexandr-shegeda shouldn't this issue be closed? if i understand correctly the destination has been labelled as GA

[alexandr-shegeda]

@sherifnada we still have open items related to the auth and data-type handling linked to this epic, should we remove them from the epic and close the issue?

[sherifnada]

@grishick to advise here -- my understanding is oauth shouldn't be part of GA but data types might be

[grishick]

@alexandr-shegeda per @misteryeo 's comment above, we can release BQ destination to GA w/o OAuth

[grishick]

I think the following issues need to be resolved to call Destination BQ GA:
https://app.zenhub.com/workspace/o/airbytehq/airbyte/issues/12655
https://app.zenhub.com/workspace/o/airbytehq/airbyte/issues/12658
https://app.zenhub.com/workspace/o/airbytehq/airbyte/issues/12660

[grishick]

Someone has already labeled BQ destination as GA