Skip to content

Destinations CDK: CatalogParser sets default namespace #38121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 10, 2024
Merged

Destinations CDK: CatalogParser sets default namespace #38121

merged 1 commit into from
Jun 10, 2024

Conversation

edgao
Copy link
Contributor

@edgao edgao commented May 10, 2024 •
edited
Loading

  • CatalogParser is now responsible for setting the default namespace on stream objects.
    • This entails a bunch of knock-on changes:
      • All the core async stuff now operates on the original (null) namespace
      • BufferEnqueue also needs to set the default namespace right before handing the message to the queue
      • AsyncStreamConsumer needs to set the default namespace when calling OnCloseFunction
    • I also took the opportunity to make defaultNamespace non-nullable.
  • This allows destinations code to stop thinking about it (i.e. we can remove all our catalog.streams.each(stream.setNamespace) blocks)

closes https://github.com/airbytehq/airbyte-internal-issues/issues/2413

@vercel Vercel
Copy link

vercel bot commented May 10, 2024 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Visit Preview Jun 10, 2024 4:13pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label May 10, 2024
@edgao Graphite App
Copy link
Contributor Author

edgao commented May 10, 2024 •
edited by gisripa
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @edgao and the rest of your teammates on Graphite Graphite

This was referenced May 10, 2024
Merged
Merged
Closed
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch 2 times, most recently from 2047aec to 2f4b6ac Compare May 10, 2024 17:33
@gisripa gisripa mentioned this pull request May 10, 2024
Merged
2 tasks
@edgao edgao changed the base branch from master to cdk_generation_id May 10, 2024 19:59
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 2f4b6ac to 08d7792 Compare May 10, 2024 19:59
@edgao edgao mentioned this pull request May 10, 2024
Merged
@edgao edgao changed the title Destinations CDK: CatalogParser extracts generation ID, sets default namespace Destinations CDK: CatalogParser extracts sets default namespace May 10, 2024
@edgao edgao changed the title Destinations CDK: CatalogParser extracts sets default namespace Destinations CDK: CatalogParser sets default namespace May 10, 2024
@edgao edgao force-pushed the cdk_generation_id branch from 0379935 to 3d4547a Compare May 10, 2024 20:23
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch 2 times, most recently from 63d10dc to 55209ad Compare May 10, 2024 20:32
@gisripa gisripa force-pushed the cdk_generation_id branch from 3d4547a to 0724d77 Compare May 10, 2024 22:05
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 55209ad to 7ac6df3 Compare May 10, 2024 22:18
This was referenced May 10, 2024
Merged
Merged
Merged
@gisripa gisripa force-pushed the cdk_generation_id branch 2 times, most recently from 0ded7a7 to 23a237f Compare May 13, 2024 15:42
Base automatically changed from cdk_generation_id to master May 13, 2024 16:59
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 7ac6df3 to 9cb852a Compare May 13, 2024 18:17
@edgao edgao mentioned this pull request May 15, 2024
Closed
2 tasks
@gisripa gisripa mentioned this pull request May 16, 2024
Merged
2 tasks
@edgao edgao force-pushed the edgao/bigquery_new_columns branch from 1b64d77 to 4590d29 Compare May 29, 2024 15:18
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch 2 times, most recently from 34069cc to d15616a Compare May 29, 2024 15:41
@edgao edgao force-pushed the edgao/bigquery_new_columns branch from 4590d29 to ae33cea Compare May 29, 2024 18:28
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from d15616a to 377fa27 Compare May 29, 2024 18:28
Base automatically changed from edgao/bigquery_new_columns to master May 29, 2024 18:43
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch 2 times, most recently from 443c89c to e29c5d9 Compare May 29, 2024 20:56
@edgao edgao changed the base branch from master to edgao/bigquery_cdk_ffs May 29, 2024 20:56
@octavia-squidington-iii octavia-squidington-iii removed the area/connectors Connector related issues label May 29, 2024
@edgao edgao mentioned this pull request May 29, 2024
Merged
Base automatically changed from edgao/bigquery_cdk_ffs to master May 30, 2024 14:48
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from e29c5d9 to 1f11d82 Compare May 30, 2024 14:49
@edgao edgao mentioned this pull request May 30, 2024
Closed
2 tasks
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 1f11d82 to afef1a6 Compare May 31, 2024 15:20
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from afef1a6 to 60fe677 Compare June 6, 2024 16:15
@gisripa gisripa force-pushed the edgao/catalog_parser_improvements branch from 60fe677 to 7910b55 Compare June 7, 2024 20:18
@gisripa gisripa mentioned this pull request Jun 7, 2024
Merged
2 tasks
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 7910b55 to 1638b6d Compare June 7, 2024 23:12
@gisripa gisripa force-pushed the edgao/catalog_parser_improvements branch from 1638b6d to 7910b55 Compare June 8, 2024 02:38
@edgao edgao force-pushed the edgao/catalog_parser_improvements branch from 7910b55 to feba86e Compare June 10, 2024 16:13
@edgao
Copy link
Contributor Author

edgao commented Jun 10, 2024 •
edited by github-actions bot
Loading

/publish-java-cdk

🕑 https://github.com/airbytehq/airbyte/actions/runs/9452125686
✅ Successfully published Java CDK version=0.37.0!

@edgao edgao merged commit a78647e into master Jun 10, 2024
25 checks passed
@edgao edgao deleted the edgao/catalog_parser_improvements branch June 10, 2024 16:40
kabeer27 pushed a commit to kabeer27/airbyte that referenced this pull request Jun 12, 2024
@edgao @kabeer27
Destinations CDK: CatalogParser sets default namespace (airbytehq#38121)
6cb7f50
@gisripa gisripa mentioned this pull request Jun 13, 2024
Merged
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gisripa gisripa gisripa approved these changes

@stephane-airbyte stephane-airbyte stephane-airbyte approved these changes

Assignees
No one assigned
Labels
CDK Connector Development Kit connectors/destination/bigquery
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@edgao @gisripa @stephane-airbyte @octavia-squidington-iii