Skip to content

run dns resolution in the same tracing-span as the caller #3870

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
joshka opened this issue Apr 2, 2025 · 2 comments
Open

run dns resolution in the same tracing-span as the caller #3870

joshka opened this issue Apr 2, 2025 · 2 comments
Labels
C-feature Category: feature. This is adding a new feature.

Comments

@joshka
Copy link

joshka commented Apr 2, 2025

Is your feature request related to a problem? Please describe.
This is an issue for hyper-util which doesn't have an issues tab

hyperium/hyper-util#134 introduced a fix with the following details:

This makes it possible to trace the entire request, including DNS
resolution.

The use case for this is to be able to suppress specific requests from
being traced in a situation where the request is used to transmit logs
to a remote server. This would result in an infinite loop of logs being
sent to the remote server. tokio-rs/tokio#6659
has more details.

But there's a tracing / tracing subscriber issue with dropping a span on a different thread than it was created when there's multiple dispatchers in play. A followup PR will revert this change hyperium/hyper-util#179

The impact of the revert will be that DNS trace logs are outside of any span that requests are made within

Describe the solution you'd like
Once the tracing issue is resolved / worked around, re-implement the fix

Describe alternatives you've considered
It might be possible to implement this instead of as a child span aligned with the current span instead as a span created for the entire DNS handling mechanism which is then never dropped.

Additional context
The overall purpose of this was to make it possible to make it easier to filter out trace messages from causing an infinite tracing loop when tracing uses a subscriber that uses hyper-util. Whatever solution that comes up should be aligned with that goal.

@markdingram
Copy link

Aside from any panic, a hyper legacy client / tower-http tracing setup started producing "long running traces unconnected with the HTTP request" after upgrading to 0.1.11.

Without digging too hard the previous change in hyperium/hyper-util#134 seems likely to have been the cause of this - see prior comment from a reqwest issue:

when a request is started, the pool that reqwest uses will create two futures, and race them: 1) ask the pool for a newly idle connection, 2) create a new connection. If the second option is started, but then the first option wins the race (especially because resolving looks like its being slow), the second future will be spawned as a background task to finish up and insert the new connection into the pool (this reduces socket thrashing).

@joshka
Copy link
Author

joshka commented Apr 3, 2025

Without having seen that, it does sound like it would probably have been related.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-feature Category: feature. This is adding a new feature.
Projects
None yet
Development

No branches or pull requests

2 participants