Skip to content

Fix data race caused by concurrent access to the request URL during scans #448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 15, 2025

Conversation

hdm
Copy link
Contributor

@hdm hdm commented Jul 15, 2025

A data race was identified when the Nuclei engine calls request.dump() when the net/http client transport for the request is still active (as part of a scan goroutine). This PR changes the Update() function to clone and replace the URL, avoiding the race. This can be tricky to reproduce (might require a HTTP/2 target) and unfortunately I misplaced the original data race log that led to this fix, but it hasn't popped up again since I made this change.

@hdm
Copy link
Contributor Author

hdm commented Jul 15, 2025

Doh, this is still triggering and will need more work, here's race log:

==================
WARNING: DATA RACE
Write at 0x00c064436540 by goroutine 867838:
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).executeRequest()
      project/nuclei/pkg/protocols/http/request.go:869 +0x7170
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults.func1()
      project/nuclei/pkg/protocols/http/request.go:508 +0xa04
  github.com/projectdiscovery/nuclei/v3/pkg/protocols/http.(*Request).ExecuteWithResults()
      project/nuclei/pkg/protocols/http/request.go:585 +0x786
  github.com/projectdiscovery/nuclei/v3/pkg/tmplexec/generic.(*Generic).ExecuteWithResults()
      project/nuclei/pkg/tmplexec/generic/exec.go:61 +0x595
  github.com/projectdiscovery/nuclei/v3/pkg/tmplexec.(*TemplateExecuter).Execute()
      project/nuclei/pkg/tmplexec/exec.go:216 +0x876
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2.1()
      project/nuclei/pkg/core/executors.go:138 +0x415
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2.gowrap1()
      project/nuclei/pkg/core/executors.go:145 +0x61

Previous read at 0x00c064436540 by goroutine 868391:
  net/http.newTransferWriter()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transfer.go:93 +0x3b3
  net/http.(*Request).write()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/request.go:706 +0x1011
  net/http.(*persistConn).writeLoop()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transport.go:2593 +0x364
  net/http.(*Transport).dialConn.gowrap3()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transport.go:1945 +0x33

Goroutine 867838 (running) created at:
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets.func2()
      project/nuclei/pkg/core/executors.go:114 +0xd6a
  github.com/projectdiscovery/nuclei/v3/pkg/input/provider.(*SimpleInputProvider).Iterate()
      project/nuclei/pkg/input/provider/simple.go:38 +0x8a
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateWithTargets()
      project/nuclei/pkg/core/executors.go:79 +0x85b
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateSpray.func1()
      project/nuclei/pkg/core/execute_options.go:137 +0x10a
  github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).executeTemplateSpray.gowrap2()
      project/nuclei/pkg/core/execute_options.go:138 +0x41

Goroutine 868391 (running) created at:
  net/http.(*Transport).dialConn()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transport.go:1945 +0x2dd9
  net/http.(*Transport).dialConnFor()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transport.go:1615 +0x11d
  net/http.(*Transport).startDialConnForLocked.func1()
      /root/go/pkg/mod/golang.org/[email protected]/src/net/http/transport.go:1597 +0x3c
==================

@hdm hdm marked this pull request as draft July 15, 2025 07:13
@hdm
Copy link
Contributor Author

hdm commented Jul 15, 2025

This PR helps, but requires a separate PR in nuclei to resolve:

@hdm hdm marked this pull request as ready for review July 15, 2025 07:44
@Mzack9999 Mzack9999 self-requested a review July 15, 2025 13:14
@Mzack9999 Mzack9999 merged commit 1e3d56a into projectdiscovery:main Jul 15, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants