Skip to content

Extend Kafka exporter to support profiling signal #37837

Open
@danielpacak

Description

@danielpacak

Component(s)

exporter/kafka

Is your feature request related to a problem? Please describe.

There is this relatively new profiling signal supported by OTel described at https://opentelemetry.io/blog/2024/state-profiling/.

It would be nice to collect and store profile samples in Kafka topics, which is currently not supported by the Kafka exporter.

Describe the solution you'd like

Add support for profiling signal so w can configure it as follows:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: ${env:MY_POD_IP}:4317
      http:
        endpoint: ${env:MY_POD_IP}:4318
exporters:
  otlp/pyroscope:
    endpoint: http://pyroscope.pyroscope:4040
    tls:
      insecure: true
  kafka/profiles:
    encoding: json
    brokers:
      - kafka.kafka:9092
    topic: my_profiles
service:
  pipelines:
    profiles:
      receivers:
        - otlp
      exporters:
        - otlp/pyroscope
        - kafka/profiles

There are options to marshal the profiling samples, but to get this feature off the ground we can use a default, opinionated marshaller just to forward messages to Kafka.

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions