Always create a JMS consumer span #10604
Conversation
|
I've seen users get tripped up by the lack of parenting for messaging/jms/kafka for sure....but do we have spec around this? Is this going to surprise some folks? |
...vaagent/src/main/java/io/opentelemetry/javaagent/instrumentation/jms/v3_0/JmsSingletons.java
Outdated
Show resolved
Hide resolved
| equalTo(SemanticAttributes.MESSAGING_OPERATION, "receive"), | ||
| equalTo(SemanticAttributes.MESSAGING_MESSAGE_ID, messageId))), | ||
| trace -> | ||
| trace.hasSpansSatisfyingExactly(span -> span.hasName("consumer parent").hasNoParent())); |
There was a problem hiding this comment.
Something is weird about this to me. We create two traces, just like we did before, but the innermost span of the second consumer trace is now part of the first trace? It seems weird now that we have two consumer spans, one rooted in a publisher parent context and another one that seems lonely ("consumer parent"). It's also no longer true that "consumer parent" is the parent of the consumer.
What am I missing?
There was a problem hiding this comment.
Thanks for looking into this or messaging instrumentation is a bit of a mess so and not the easiest to understand. We'd like all of our messaging instrumentations to follow the same rules, but unfortunately this is not the case. Currently I think kafka and sqs instrumentations are probably in the best shape.
The telemetry that the messaging instrumentations create depends on the otel.instrumentation.messaging.experimental.receive-telemetry.enabled flag, which by default is false. This is tested in the Jms3SuppressReceiveSpansTest, true is tested in Jms3InstrumentationTest. When this flag is true we should create telemetry where producer and consumer are in different traces that are connected via a span link. Having consumer span for receive operation being parent of process span is for example described in https://github.com/open-telemetry/semantic-conventions/blob/v1.21.0/docs/messaging/messaging-spans.md#batch-receiving Spec has been using span links for a while and at one point we even turned this on by default, but had to revert because at that time most backends did not support the span links. Our default behavior is to place producer and consume in the same trace so that producer is the parent of the consumer and there are no span links. When using this strategy we usually don't create consumer spans for receive operation and only have process spans (this motivated the receive-telemetry in the flag name). If there is no process span then there will be no consumer spans at all, which is what this PR tries to address (or initially tried to address, had to make more changes to get spring jms tests passing too).
There was a problem hiding this comment.
Thanks for clarifying, appreciate it.
...ent/src/main/java/io/opentelemetry/javaagent/instrumentation/jms/JmsInstrumenterFactory.java
Outdated
Show resolved
Hide resolved
Currently when using
MessageConsumer#receive()withreceive-telemetrydisabled (default) we don't create any spans on the consumer side. This PR changes this behavior so that areceivespan is created that is the child of the producer.