Skip to content

[airbyte-cdk] entrypoint wrapper should use per-stream state not legacy format #35976

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 11, 2024

Conversation

brianjlai
Copy link
Contributor

@brianjlai brianjlai commented Mar 11, 2024

After deprecating legacy state from the CDK, I didn't update the CDK mock server test wrapper to refer to the per-stream state format instead of deprecated legacy data field. This is blocking updating some of our concurrent connectors that have mocker server tests since we're asserting on a field that is no longer populated.

@brianjlai brianjlai requested review from erohmensing and a team March 11, 2024 22:12
Copy link

vercel bot commented Mar 11, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated (UTC)
airbyte-docs ⬜️ Ignored (Inspect) Mar 11, 2024 10:12pm

@octavia-squidington-iii octavia-squidington-iii added the CDK Connector Development Kit label Mar 11, 2024
@brianjlai brianjlai changed the title entrypoint wrapper should use per-stream state not legacy format [airbyte-cdk] entrypoint wrapper should use per-stream state not legacy format Mar 11, 2024
Copy link
Contributor

@erohmensing erohmensing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems sane! Can you point to where things start failing (since they didn't start failing when the code was released?)

Is it that we are just now getting all the old-state-emitters up to date, and if so, are there any that we can't feasibly update in time that will start failing as a result? do we need to be able to handle both in an interim state?

(mostly questions from a learning standpoint and to make sure we've covered our bases - no real concerns as this is test code, easily revertible and you know way more than I do about it)

@brianjlai
Copy link
Contributor Author

@erohmensing

Can you point to where things start failing (since they didn't start failing when the code was released?)

Sure, so taking this Stripe mock server test as an example. We assert on that map which will 100% fail because we try to get the data field instead of the stream field.

Is it that we are just now getting all the old-state-emitters up to date, and if so, are there any that we can't feasibly update in time that will start failing as a result? do we need to be able to handle both in an interim state?

Not that I'm aware of. I just opted to get rid of legacy because it wasn't being used anymore. We shouldn't need to worry about interim state because the platform/destinations are operating on per-stream. The only impact was connectors that used mock-server tests because the bug was in the test code.

@brianjlai brianjlai merged commit b1f3b8a into master Mar 11, 2024
29 checks passed
@brianjlai brianjlai deleted the brian/fix_state_message_entrypoint_wrapper branch March 11, 2024 23:09
@brianjlai brianjlai restored the brian/fix_state_message_entrypoint_wrapper branch March 12, 2024 00:35
@brianjlai brianjlai deleted the brian/fix_state_message_entrypoint_wrapper branch March 12, 2024 00:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CDK Connector Development Kit
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants