Add custom javaagent <-> application context bridge to enable custom context propagation

[jmferland]

Is your feature request related to a problem? Please describe.

I'd like to leverage OTel context propagation to propagate my own custom Context scoped-values, similar to io.opentelemetry.api.baggage.Baggage. However, I do not want to use Baggage because the custom values are their own separate logical set of data with their own specific constraints whose propagation I'd like to independently enable or disable (e.g. -Dotel.propagators=tracecontext,baggage,custom or -Dotel.propagators=tracecontext,custom).

I'd also like to read my custom values from an application's Context.current(), which were extracted from a custom header by a custom propagator via an OTel Java agent extension.

However, this doesn't seem to work because the Java agent context is separate from the application context, except for Span and Baggage, which have a ContextKeyBridge.

Describe the solution you'd like

Therefore, I'd like to be able to register a custom ContextKeyBridge in AgentContextWrapper.CONTEXT_KEY_BRIDGES, perhaps similar to how custom propagators are registered, so that my custom Context scoped-values can be shared between agent and application.

Describe alternatives you've considered

1. Just use Baggage; however, Baggage allows for any data to be set and propagated. I only want a small set of data with very specific constraints (e.g. a UUID and an Instant).
2. It's theoretically possible to (ab)use Baggage to pass data between the Java agent and application contexts; however, it would be messy, leaky, and have unwanted side effects.

Additional context

No response

[trask]

(just linking this to open-telemetry/oteps#207 so we can continue to gather use cases for that enhancement proposal)

[laurit]

For your use case it would probably be simpler if instead of using the context key directly you had an api. For example

class Foo {
  static final ContextKey<X> KEY = ContextKey.named("xxx");

  static X get(Context context) {
    return context.get(KEY);
  }

  static Context set(Context context, X x) {
    return context.with(KEY, x);
  }
}

Then you could instrument the get and set methods. For example you could transform get to something like

  static X get(Context applicationContext) {
    agent.Context agentContext = AgentContextStorage.getAgentContext(applicationContext);
    AgentX agentValue = agentContext.get(AGENT_KEY);
    return applicationX(agentValue);
  }

There the problematic part is getting the AGENT_KEY that will probably require some clever use of reflection. Depending on they type of the value you may need to translate between agent and application view of the value, this may also require reflection.

Therefore, I'd like to be able to register a custom ContextKeyBridge in AgentContextWrapper.CONTEXT_KEY_BRIDGES, perhaps similar to how custom propagators are registered, so that my custom Context scoped-values can be shared between agent and application.

The problem is that this bridging happens in the application code so the bridge would also be need to registered from application code. We probably would not want to commit to having a stable api for this but if a prototype and tests are provided where a public class that exposes a way to add context bridges is introduced next to AgentContextWrapper I think we could consider merging it.

[srstrickland]

hi @jmferland, I stumbled across this thread because I'm trying to do basically the same thing, and was surprised to find that I cannot (at least, not easily). Just wondering if you found a solution?

I'm trying to set my own context attribute, and retrieve it later in a custom LogRecordProcessor/SpanProcessor (pretty much identical to what's mentioned above). What's frustrating is I can see the data in the debugger, but it's inaccessible because it's stuck behind an "otel-context" key, values can only be retrieved by reference to the ContextKey (not by value), and I don't have a way to access it.