[Issue-9926][Pulsar Functions] Pass through record properties from Pulsar Sources #9943
Conversation
| if (this.instanceConfig.getFunctionDetails().getComponentType()== ComponentType.SOURCE) { | ||
| // Forward record level properties for all sources by default | ||
| pulsarSinkConfig.setForwardSourceMessageProperty(true); | ||
| } else if (this.instanceConfig.getFunctionDetails().getComponentType() == ComponentType.SINK) { |
There was a problem hiding this comment.
Can this be a simple else ? ( without if)
There was a problem hiding this comment.
Unfortunately it cannot because there are 4 different component type enums:
enum ComponentType {
UNKNOWN = 0;
FUNCTION = 1;
SOURCE = 2;
SINK = 3;
}
There was a problem hiding this comment.
A more comprehensive solution would be to add a ForwardSourceMessageProperty to the configuration for ALL component types and use a default value of true for all types. Seems like from a consistency perspective having all the components behave exactly the same w.r.t to forward properties but leave an option for the user to override this behavior as needed would be a good approach from an end-user experience perspective.
Thoughts?
There was a problem hiding this comment.
@eolivelli I have replaced the code with a different approach. If you have time can you please review the updated code and mark this issue as resolved or suggest what other changes are required to address you concerns? Thanks in advance!
|
/pulsarbot run-failure-checks |
david-streamlio
force-pushed
the
Issue-9926
branch
from
9a6553a to
8dab7b4
Compare
jerrypeng
added
area/connector
type/bug
|
/pulsarbot run-failure-checks |
|
/pulsarbot run-failure-checks |
|
/pulsarbot run-failure-checks |
1 similar comment
|
/pulsarbot run-failure-checks |
…lsar Sources (#9943) Co-authored-by: David Kjerrumgaard <[email protected]> (cherry picked from commit 1273c71)
…lsar Sources (apache#9943) Co-authored-by: David Kjerrumgaard <[email protected]>
No description provided.