Concurrent CDK: if exception is AirbyteTracedException, raise this an… #37443
Conversation
…d not StreamThreadException
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
|
|
||
| stream_descriptor = StreamDescriptor(name=exception.stream_name) | ||
| if isinstance(exception.exception, AirbyteTracedException): | ||
| yield exception.exception.as_airbyte_message(stream_descriptor=stream_descriptor) |
There was a problem hiding this comment.
I'm not sure about how I feel by passing stream_descriptor as part of as_airbyte_message. On one hand, we mentioned it was probably better to deprecate. On the other, I'm not sure all AirbyteTracedException have the context of the stream and hence might benefit from the added information
There was a problem hiding this comment.
I can live w/ it still being there. I agree that the stream descriptor isn't applicable for failures outside of the specific stream being processed. But for the purposes of this current ticket, we can move forward with using the signature as it's already declared. We're not changing things in a negative way
|
|
||
| stream_descriptor = StreamDescriptor(name=exception.stream_name) | ||
| if isinstance(exception.exception, AirbyteTracedException): | ||
| yield exception.exception.as_airbyte_message(stream_descriptor=stream_descriptor) |
There was a problem hiding this comment.
I can live w/ it still being there. I agree that the stream descriptor isn't applicable for failures outside of the specific stream being processed. But for the purposes of this current ticket, we can move forward with using the signature as it's already declared. We're not changing things in a negative way
…d not StreamThreadException
What
Following this change we've seen that blindly creating a traced exception using the StreamThreadException leads to trace message that are system_errors instead of the underlying error type.
How
Checking the type of the exception and if it is already
AirbyteTracedException, emit it as-isUser Impact
Syncs failing like with config_errors will not be flagged as system_error (see this for more information).
Can this PR be safely reverted and rolled back?