Skip to content

[receiver/datadogreceiver] Configure /intake endpoint #39787

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

goakley
Copy link
Contributor

@goakley goakley commented Apr 30, 2025

Description

Datadog's /intake endpoint is used by agents to record information about the system they're running on. It's not a metrics/logs/traces endpoint, but instead accepts metadata about a specific host that a datadog-agent is running on. This meatadata is used by Datadog to enrich metrics for a specific host (e.g. host-specific tags that should be associated with all metrics from that host).

While supporting /intake isn't striclty necessary to ingest metrics/logs/traces, it means that pointing a datadog-agent at an OpenTelemetry Collector will not work as expected, because all metrics submitted by the agent will be missing their host tags. By adding the ability to proxy /intake calls to Datadog itself, we enable this datadog-agentotelcol workflow.

The default behaviour maintains backwards-compatibility - if you don't specify an intake configuration, then intake calls aren't processed.

Testing

Ran both simulated calls and ran a datadog-agent against this receiver. The collector properly received and proxied the /intake calls (as could be seen in the Datadog hosts dashboard).

Documentation

Updated the README for the receiver.

@goakley goakley requested review from MovieStoreGuy and a team as code owners April 30, 2025 15:39
Copy link

linux-foundation-easycla bot commented Apr 30, 2025

CLA Signed

The committers listed above are authorized under a signed CLA.

@atoulme
Copy link
Contributor

atoulme commented May 2, 2025

Please sign the CLA.

@goakley
Copy link
Contributor Author

goakley commented May 2, 2025

It is a WIP, I'll sign it as soon as I have approval.

@goakley
Copy link
Contributor Author

goakley commented May 13, 2025

@atoulme CLA signed!

@MovieStoreGuy
Copy link
Contributor

Please address the build issues.

Comment on lines 517 to 518
req.Header.Del("Dd-Api-Key")
req.Header.Add("Dd-Api-Key", key)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not use Set here? It should replace what currently exists and saves needed to delete the entry?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very true. This uses Set now.

@goakley
Copy link
Contributor Author

goakley commented May 14, 2025

Can someone tell me how to retrigger the checks? They don't seem to be queued in GitHub Actions.

@goakley
Copy link
Contributor Author

goakley commented May 20, 2025

@MovieStoreGuy please take another look. There are conflicts, but I'm not sure if it's appropriate to rebase before the review completes.

Copy link
Contributor

github-actions bot commented Jun 4, 2025

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Jun 4, 2025
@goakley goakley force-pushed the datadogreceiver/intake-endpoint branch from 922cead to 13e1832 Compare June 9, 2025 20:01
@github-actions github-actions bot removed the Stale label Jun 10, 2025
@goakley goakley force-pushed the datadogreceiver/intake-endpoint branch 2 times, most recently from ac752fc to f0be05e Compare June 12, 2025 17:56
@goakley goakley force-pushed the datadogreceiver/intake-endpoint branch from f0be05e to a164481 Compare June 16, 2025 19:00
@atoulme atoulme added ready to merge Code review completed; ready to merge by maintainers and removed waiting-for-code-owners ready to merge Code review completed; ready to merge by maintainers labels Jun 16, 2025
@atoulme atoulme marked this pull request as draft June 16, 2025 20:52
@atoulme
Copy link
Contributor

atoulme commented Jun 16, 2025

Please review the CI build, address any issues, and mark ready for review again.

@MovieStoreGuy MovieStoreGuy marked this pull request as ready for review June 20, 2025 06:33
goakley added 7 commits June 20, 2025 17:01
Datadog's `/intake` endpoint is used by agents to record information about the system they're running on.
It's not a metrics/logs/traces endpoint, but instead accepts metadata about a specific host that a `datadog-agent` is running on.
This meatadata is used by Datadog to enrich metrics for a specific host (e.g. host-specific tags that should be associated with all metrics from that host).

While supporting `/intake` isn't striclty necessary to ingest metrics/logs/traces, it means that [pointing a `datadog-agent` at an OpenTelemetry Collector](https://github.com/DataDog/datadog-agent/blob/7.64.0/pkg/config/config_template.yaml#L40) will not work as expected, because all metrics submitted by the agent will be missing their host tags.
By adding the ability to proxy `/intake` calls to Datadog itself, we enable this `datadog-agent`→`otelcol` workflow.
@goakley goakley force-pushed the datadogreceiver/intake-endpoint branch from bd3832a to 43ef34c Compare June 20, 2025 17:13
@MovieStoreGuy MovieStoreGuy merged commit ba9f4b4 into open-telemetry:main Jun 23, 2025
177 checks passed
@github-actions github-actions bot added this to the next release milestone Jun 23, 2025
@goakley goakley deleted the datadogreceiver/intake-endpoint branch June 26, 2025 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants