Skip to content

[feature request] [Exporter.Geneva] Handle all-numeric attribute names without corruption #2717

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Walter-Rector opened this issue Apr 26, 2025 · 3 comments
Labels
comp:exporter.geneva Things related to OpenTelemetry.Exporter.Geneva enhancement New feature or request

Comments

@Walter-Rector
Copy link

Component

OpenTelemetry.Exporter.Geneva

Is your feature request related to a problem?

Yes:

We've recently migrated over to using OpenTelemetry/Geneva Exporter in Linux containers from a much older system using MonAgent in windows containers. We started seeing periods in our logs where DGREP reported "Found corrupted logs in storage", making logs during those times unavailable.

After a lengthy investigation we discovered that one of the external components was using positional parameters in the ILogger format string instead of named parameters. For example

logger.LogInformation("Init of {0} {1} on silo {2} for queue {3}.",
    GetType().Name,
    ((ISystemTargetBase)this).GrainId.ToDetailedString(),
    Silo,
    QueueId.ToStringWithHashCode());

This caused the attributes to be named "0", "1", "2", etc. When attributes with these names are sent to Geneva, they cause the "Found corrupted logs in storage" message for the entire block of logs in the blob.

I wrote a Processor that will rename any attributes who's name can convert to an int to a standard alpha-numeric name, so we have a workaround, but it would be great if the Geneva Exporter could do one of the following:

  • Write the attribute name in a way that doesn't cause corruption (like adding some standard prefix like "_" or anything that geneva will accept).
  • Log to the open-telemetry ETW logs when experiencing this issue.
  • Drop the log entirely to avoid corrupting the surrounding logs

Totally open to other ideas as well

What is the expected behavior?

Numeric attribute names don't cause the 'Found corrupted logs in storage"

Which alternative solutions or features have you considered?

I've built a custom otel Processor that renames the attributes when they are convertible to an int.

This works for us, but trying to troubleshoot this issue was very time consuming and it would be helpful to other users to either have not be a problem or to warn them what is going on.

Additional context

No response

@Walter-Rector Walter-Rector added the enhancement New feature or request label Apr 26, 2025
@github-actions github-actions bot added the comp:exporter.geneva Things related to OpenTelemetry.Exporter.Geneva label Apr 26, 2025
Copy link
Contributor

Tagging component owner(s).

@rajkumar-rangaraj @TimothyMothra @xiang17

@harsimar
Copy link

harsimar commented May 1, 2025

Hi,
The general guidance when logging in dotnet is to use compile time logging source generation - other methods are less performant. See: open telemetry logging docs & dotnet docs & example code.

While that log processor may work in the short term, please understand that using positional parameters is not a recommended practice and is outdated. It is unlikely that we will be making any code change to support a less performant scenario.

@andrew-hampton
Copy link

As noted, above these logging calls are coming from an external component via nuget package that we're consuming and not code directly within our control.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp:exporter.geneva Things related to OpenTelemetry.Exporter.Geneva enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants