Make neighbors routing tables access mutual exclusive

[drwtsn95]

Fixed dataraces encountered when --neighbors=true.

[dennis-tra]

Hi @drwtsn95, thanks for the PR. Have you run into this concurrent map access issue yourself? I'm asking because Flush is strictly called after all visits were inserted so there should not be a concurrent access. However, if you've encountered the error, then clearly something's wrong.

[drwtsn95]

Hi! I have encountered it into InsertVisit function (nebula/db/pg.go:643) but then mutexed all the accesses for consistency, so doesn't even know that Flush is called without concurrency, making it code-aware-agnostic :D
I've also tried to preserver your concurrent map handling style, but initially thought about sync.Map or xsync.MapOf.

[drwtsn95]

WARNING: DATA RACE
Write at 0x00c0007915c0 by goroutine 5106:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:223 +0x0
  github.com/dennis-tra/nebula-crawler/db.(*PostgresClient).InsertVisit()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/db/pg.go:643 +0x81b
  github.com/dennis-tra/nebula-crawler/core.(*CrawlWriter[go.shape.struct { *github.com/ethereum/go-ethereum/p2p/enode.Node; github.com/dennis-tra/nebula-crawler/discv4.peerID github.com/libp2p/go-libp2p/core/peer.ID; github.com/dennis-tra/nebula-crawler/discv4.maddrs []github.com/multiformats/go-multiaddr.Multiaddr; github.com/dennis-tra/nebula-crawler/discv4.udpAddr net/netip.AddrPort; github.com/dennis-tra/nebula-crawler/discv4.enr string; github.com/dennis-tra/nebula-crawler/discv4.udpIdx int8; github.com/dennis-tra/nebula-crawler/discv4.tcpIdx int8 }]).Work()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/writer_crawl.go:99 +0x12cc
  github.com/dennis-tra/nebula-crawler/core.(*CrawlWriter[github.com/dennis-tra/nebula-crawler/discv4.PeerInfo]).Work()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/writer_crawl.go:38 +0xb6
  github.com/dennis-tra/nebula-crawler/core.(*Engine[go.shape.struct { *github.com/ethereum/go-ethereum/p2p/enode.Node; github.com/dennis-tra/nebula-crawler/discv4.peerID github.com/libp2p/go-libp2p/core/peer.ID; github.com/dennis-tra/nebula-crawler/discv4.maddrs []github.com/multiformats/go-multiaddr.Multiaddr; github.com/dennis-tra/nebula-crawler/discv4.udpAddr net/netip.AddrPort; github.com/dennis-tra/nebula-crawler/discv4.enr string; github.com/dennis-tra/nebula-crawler/discv4.udpIdx int8; github.com/dennis-tra/nebula-crawler/discv4.tcpIdx int8 },go.shape.7585e8597360114ba4a7b52ada9db167452cf8d48d094994bf5b1318f62ec571]).Run.(*Pool[go.shape.7585e8597360114ba4a7b52ada9db167452cf8d48d094994bf5b1318f62ec571,go.shape.struct { WriterID string; PeerID github.com/libp2p/go-libp2p/core/peer.ID; Duration time.Duration; Error error }]).Start.func2.1()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/worker.go:34 +0x137

Previous write at 0x00c0007915c0 by goroutine 5107:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:223 +0x0
  github.com/dennis-tra/nebula-crawler/db.(*PostgresClient).InsertVisit()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/db/pg.go:643 +0x81b
  github.com/dennis-tra/nebula-crawler/core.(*CrawlWriter[go.shape.struct { *github.com/ethereum/go-ethereum/p2p/enode.Node; github.com/dennis-tra/nebula-crawler/discv4.peerID github.com/libp2p/go-libp2p/core/peer.ID; github.com/dennis-tra/nebula-crawler/discv4.maddrs []github.com/multiformats/go-multiaddr.Multiaddr; github.com/dennis-tra/nebula-crawler/discv4.udpAddr net/netip.AddrPort; github.com/dennis-tra/nebula-crawler/discv4.enr string; github.com/dennis-tra/nebula-crawler/discv4.udpIdx int8; github.com/dennis-tra/nebula-crawler/discv4.tcpIdx int8 }]).Work()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/writer_crawl.go:99 +0x12cc
  github.com/dennis-tra/nebula-crawler/core.(*CrawlWriter[github.com/dennis-tra/nebula-crawler/discv4.PeerInfo]).Work()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/writer_crawl.go:38 +0xb6
  github.com/dennis-tra/nebula-crawler/core.(*Engine[go.shape.struct { *github.com/ethereum/go-ethereum/p2p/enode.Node; github.com/dennis-tra/nebula-crawler/discv4.peerID github.com/libp2p/go-libp2p/core/peer.ID; github.com/dennis-tra/nebula-crawler/discv4.maddrs []github.com/multiformats/go-multiaddr.Multiaddr; github.com/dennis-tra/nebula-crawler/discv4.udpAddr net/netip.AddrPort; github.com/dennis-tra/nebula-crawler/discv4.enr string; github.com/dennis-tra/nebula-crawler/discv4.udpIdx int8; github.com/dennis-tra/nebula-crawler/discv4.tcpIdx int8 },go.shape.7585e8597360114ba4a7b52ada9db167452cf8d48d094994bf5b1318f62ec571]).Run.(*Pool[go.shape.7585e8597360114ba4a7b52ada9db167452cf8d48d094994bf5b1318f62ec571,go.shape.struct { WriterID string; PeerID github.com/libp2p/go-libp2p/core/peer.ID; Duration time.Duration; Error error }]).Start.func2.1()
      /home/drwtsn/web3/arbitrage/polygon/p2p/nebula/core/worker.go:34 +0x137

[dennis-tra]

Ah this makes sense now! There are multiple workers writing visits into the DB and every single one is appending to the map concurrently. Guarding the Flush part isn't strictly necessary but certainly good practice!

Thanks for the patch and for reporting!