Skip to content

Apply buffering changes to BigQuery Destination when using staging #11203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
4 tasks done
sherifnada opened this issue Mar 16, 2022 · 1 comment · Fixed by #11776
Closed
4 tasks done

Apply buffering changes to BigQuery Destination when using staging #11203

sherifnada opened this issue Mar 16, 2022 · 1 comment · Fixed by #11776
Assignees
@tuliren
Labels
area/connectors Connector related issues area/reliability area/warehouses connectors/destinations-warehouse

Comments

@sherifnada
Copy link
Contributor

sherifnada commented Mar 16, 2022 •
edited by tuliren
Loading

Subtasks

  • Refactor GCS and S3 related classes to DRY the code
    • S3 and GCS destination config
  • Migrate BigQuery destination to use the new serialized buffering staging strategy
  • Migrate BigQuery denormalized destination to use the new serialized buffering staging strategy
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tuliren tuliren

Labels
area/connectors Connector related issues area/reliability area/warehouses connectors/destinations-warehouse
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

🎉 BigQuery destination: use serialized buffer for gcs staging airbytehq/airbyte
4 participants
@bleonard @tuliren @sherifnada @ChristopheDuong