feat(csharp/src/Apache.Arrow.Adbc): OpenTelemetry tracing baseline

[birschick-bq]

This PR adds OpenTelemetry-compatible tracing support on the CSharp library.

Under the hood it uses Activity and ActivitySource from the net System.Diagnostics.DiagnosticSource package.

Inherit from TracingConnection

public class YourConnection: TracingConnection
{
    public YourConnection(IReadOnlyDictionary<string, string> properties)
        : base(properties)
    {
    }
}

By default, the activitySourceName will be created from the driver assembly name.

Then to instrument tracing, use one of the following overrides of

• TraceActivity
• TraceActivityAsync

Example:

 public void Connect(...)
 {
    TraceActivity((activity) =>
    {
        driver.OpenSession(...);
    });
 }

Each of these overrides starts a new Activity which will be non-null only if there is an active ActivityListener or OpenTelemetry exporter. The Activity is passed to the delegate Func/Action in case it need to add ActivityEvent, ActivityLink or Tags (KeyValuePair). When instrumenting tracing, you should always use the null-conditional operator (?. ) when accessing members on the passed delegate parameter, activity.

Example:

public IArrowArrayStream GetObjects(...)
{
    return TraceActivity((activity) =>
    {
        activity?.AddEvent("db.operation.name.start", [new("db.operation.name", nameof(Client.GetCatalogs))]);
        var result = driver.GetCatalogs(...);
        var eventActivity = activity?.AddEvent("db.operation.name.end",
            [
                new("db.operation.name", nameof(Client.GetCatalogs)),
                new("db.response.status_code", getCatalogsResp.Status.StatusCode.ToString())
            ]);
        return ...
    });
}

The default behavior is to invoke the delegate and if successful (i.e., no exception thrown), the Activity.SetStatus is set to Ok. If an exception is observed, then the Activity.SetStatus is set to Error and the exception is traced (Activity.AddException) as an event in the current Activity.

Callers can pass a traceparent string to any of the TraceActivity[Async] methods using the optional traceParent parameter. The parameter takes precedence over the property. The traceId from the traceParent parameter or TraceParent property will be adopted as the rootId for all trace Activity on that call or object. If TraceParent is null (initial or set later), then the Activity creates a new rootId for the beginning of the initial Activity in the stack.

Example:

public void Connect(..., string? traceParent = default)
{
    TraceActivity((activity) =>
    {
        activity?.AddEvent("opensession.start");
        var result = driver.OpenSession(...);
        activity?.AddEvent("opensession.end", ["status", result.Status]);
    }, traceParent: traceParent);
}

Identifiers used for events and tags should follow the OpenTelemetry semantic guidance ..
https://opentelemetry.io/docs/specs/semconv/database/database-spans/
https://opentelemetry.io/docs/specs/semconv/database/database-metrics/

[CurtHagenlocher]

Thanks! I'm not done reviewing this yet but I've done about three hours of code reviews so far today and need a break... .

[CurtHagenlocher]

Hmm... so I guess this is where the parent is set? And the scenario is that the caller opens a connection once but can set a new parent every time the execute a statement?

[CurtHagenlocher]

It seems strange to allow the parent to be set independently for the connection and statement, and for these to have no relationship to each other. But maybe my mental model for the parent isn't right?

[birschick-bq]

@CurtHagenlocher

So now, only allow the trace parent to be set on the Connection - either at construction or anytime after. Disallowing setting the trace parent on Statement.

[birschick-bq]

@CurtHagenlocher - Removed the exporter components. I will create another PR to have exporter(s) as separate project/package.