Unable to generate deterministic IDs for use during a workflow run

[dustyhorizon]

Component(s)

receiver/github

What happened?

Description

Per the new*ID functions, I am trying to replicate the creation of deterministic Span IDs to be used within the workflows itself i.e. creating spans of command executions via otel-cli

However, I am having issues trying to derive the various parameters used by the GitHub receiver i.e. newJobSpanID uses event.GetWorkflowJob().GetName() which is not readily available within an GitHub actions environment, similar issue with newStepSpanID where there is something about step names and step numbers.

Would it be possible to change the parameters used by these new*ID functions to use variables that are available within a GitHub actions environment during a workflow run? Maybe we could use a combination of JobID/StepID given that every job/step has to have an ID in the workflow, and that can be easily derived since we have to specify the ID in the file anyway.

This will make collected traces cleaner and sensible such as having otel-cli spans having the correct ancestry i.e. such command runs should be a span under a workflow step.

Steps to Reproduce

N/A

Expected Result

Being able to generate deterministic IDs during a workflow run in GitHub actions, with existing variables/contexts

Actual Result

Unable to generate deterministic IDs during a workflow run in GitHub actions due to variables used by the GitHub receiver being unavailable within the running environment.

Collector version

1a91389

Environment information

Environment

CI: GitHub Actions

OpenTelemetry Collector configuration

Log output

Additional context

No response

[github-actions]

Pinging code owners:

• receiver/github: @adrielp @andrzej-stencel @crobert-1 @TylerHelmuth

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[adrielp]

@dustyhorizon - Thanks for reaching out. The original precursor to this implementation had a corresponding action that was built for this very thing: the run-with-telemetry action. It "should" still work, though it'd be great for us to have an officially supported version in the future.

There should be enough information between environment variables and context within workflows to do the manual instrumentation, though that may turn into a bit of bash code to generate the ID's etc.

I've got a lot of things I'm working on prior to kubecon, one of my stretch goals is to have a working gist that would allow this to be done manually without actions, but I haven't had the time to make that happen yet. If you'd like to message me in the CNCF Slack, I'd be happy to assist you if you choose to go that route.

[jomcgi]

Hey! is there anything I can do to help?
I was building a go binary & using that to output the trace env variables in CI for the step.
Then parsing them in my code

[adrielp]

hey @jomcgi - sorry for the delay in response. Catching up after Kubecon.

I would love to sync up on next steps for this. Heard about a new project in the space today as well that highly relates thoth. Might be something we can use/partner with a little bit so definitely want to figure out whats next.

It'd be great to have a defacto way within OTel that we support/provide like the GitHub Receiver. But also, there's the possibility of talks with GitHub directly for instrumenting the runners themselves so that we don't have have a receiver (other than OTLP) nor import specific actions.

I'm on CNCF slack in the #otel-cicd channel if you want to do some quick brainstorming, and this would be a great topic to bring up during the next CICD SIG call next Thursday.

[dustyhorizon]

I realized that there are some limitations in generating trace ids using via values from github contexts (within the workflow) and values from github API such as job names (especially in reusable workflows or matrix builds) is not available via contexts and that you have to interpolate it yourself etc.

Thanks for sharing thoth, I was leveraging otel-cli but it seems to be a bit outdated although it still works ... will be exploring thoth as a replacement 😅

[adrielp]

I realized that there are some limitations in generating trace ids using via values from github contexts (within the workflow) and values from github API such as job names (especially in reusable workflows or matrix builds) is not available via contexts and that you have to interpolate it yourself etc.

Thanks for sharing thoth, I was leveraging otel-cli but it seems to be a bit outdated although it still works ... will be exploring thoth as a replacement 😅

My pleasure! I haven't done a deep dive on Thoth yet, so I'm not entirely sure if it'll provide the exact functionality to make things work, but I definitely plan on looking at it myself.

Also, you may be interested in this thread which highly relates to this conversation @dustyhorizon.