Feature Request: Support for Adding OpenTelemetry Span Events with Dynamic Attributes

[alessandrobologna]

Feature Request: Support for Adding OpenTelemetry Span Events with Dynamic Attributes

Motivation

Currently, the primary way to add span events within tracing-opentelemetry is via the tracing::event! macro. While this works well for statically defined attributes, it presents a limitation when needing to record span events where attribute keys are determined at runtime.

Use Case: Processing arbitrary structured data (e.g., JSON payloads for configuration updates, user interactions) where the specific fields within the payload are not known at compile time. The goal is to record these dynamic key-value pairs as distinct OpenTelemetry span event attributes so that they can be used for querying and filtering in the various observability backends.

Problem: The tracing::event! macro requires static field names. The only workaround for dynamic keys is to serialize the attributes (e.g., to JSON) into a single string field. This is often not optimal, as it may prevent or make it more difficult to do filtering, aggregation, or searching based on the actual dynamic keys within the backend system. It also makes it more difficult to port otel instrumented code from other languages to rust, or creating internal libraries that provide similar features across platforms.

Proposal

I propose extending the OpenTelemetrySpanExt trait with methods to directly add OpenTelemetry span events, mirroring the flexibility of the underlying OpenTelemetry API:

trait OpenTelemetrySpanExt {
    // ... existing methods ...

    fn add_otel_span_event(&self, name: String, attributes: Vec<KeyValue>);
    fn add_otel_span_event_with_timestamp(&self, name: String, timestamp: SystemTime, attributes: Vec<KeyValue>);
}

The trait methods could be just called add_event for consistency with the otel Span trait, but maybe that could be confusing with tracing event, so I thought a more specific name may make more sense in this context.

Implementation: These methods would leverage the existing with_subscriber -> downcast_ref::<WithContext> -> with_context pattern (similar to set_attribute and set_status) to access the internal OtelData for the span and append a constructed opentelemetry::trace::Event to its builder.events list.

Why: This provides a direct, idiomatic way within the tracing-opentelemetry ecosystem to handle span events with dynamic attributes, avoiding lossy serialization. It aligns with the precedent set by set_attribute for handling dynamic span data.

Drawbacks: Minimal drawbacks anticipated. It adds two methods to the trait surface area. It relies on the with_subscriber mechanism working correctly in the user's setup.

Alternatives

1. tracing::event! + Serialization:
   • Drawback: As mentioned, serializing dynamic attributes into a string significantly harms observability and querying capabilities.
2. Manual OpenTelemetry Context Management:
   • Drawback: Managing OpenTelemetry context manually alongside tracing context is complex, error-prone, and would certainly defeat much of the purpose of using tracing-opentelemetry.

I think the addition to OpenTelemetrySpanExt is preferred because it directly addresses the limitation for dynamic attributes within the existing crate structure, is consistent with other extension methods like set_attribute, or add_link and avoids the major drawbacks of the alternatives.

(I have a working implementation of this in a local fork and would be happy to open a PR if this feature request is accepted.)

[djc]

The tracing-opentelemetry crate is currently pretty low on maintenance resources. While I'm sympathetic to your use case and wouldn't mind reviewing your PR if it's not too large, I'm also aware of some efforts in the opentelemetry-rust community to work on a better integration between tracing and the opentelemetry crates. If you intend to build something future proof (rather than a short-term solution), it might make sense to figure out whether your solution fits in with their vision.

[alessandrobologna]

Completely understand, and yes, I am following closely what's happening in the opentelemetry-rust community, but as of today I don't think there's a replacement for what this crate does (integrating the existing Tokio tracing instrumented crates with OTel). There's something for logs, but not for span and traces. And there are, as of last count, 236 other crates that take advantage of this integration.
I am going to push a PR later today, it's a small one.