bug: local node missing from /routing/v1/providers results

[SgtPooki]

Checklist

• This is a bug report, not a question. Ask questions on discuss.ipfs.tech.
• I have searched on the issue tracker for my bug.
• I am running the latest kubo version or have an issue updating.

Installation method

ipfs-desktop

Version

0.28.0 
https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/blob/5fa8b9cfad9b7f5b0a991d7efc58062f824591d8/package.json#L17

Config

config is via ipfsd-ctl in https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/blob/5fa8b9cfad9b7f5b0a991d7efc58062f824591d8/index.js#L16-L40

  {
    kuboRpcModule: kuboRpcClient,
    ipfsBin: kuboPath(),
    // test: true,
    ipfsOptions: {
      config: {
        Addresses: {
          Swarm: [
            '/ip4/0.0.0.0/tcp/4001',
            '/ip4/0.0.0.0/tcp/4002/ws'
          ],
          Gateway: '/ip4/127.0.0.1/tcp/8180'
        },
        Gateway: {
          NoFetch: true,
          ExposeRoutingAPI: true,
          HTTPHeaders: {
            'Access-Control-Allow-Origin': ['*'],
            'Access-Control-Allow-Methods': ['GET', 'POST', 'PUT', 'OPTIONS']
          }
        }
      }
    },
    args: ['--enable-pubsub-experiment', '--enable-namesys-pubsub']
  }

Description

Repro at https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/

Basically, if I enable ExposeRoutingAPI, and NoFetch, I should at least get the local kubo node to return itself as a router.

Another question here: is there any scenario where respecting NoFetch makes sense with routing/v1 api? i.e. not querying the network for providers, only returning providers we know about? If locally configured providers makes sense at all, do we need a second config option, or can we overload NoFetch?

cc @hacdias @lidel

[lidel]

Not returning self as provider on /routing/v1 indeed ended up looking like a bug.
This is because /routing/v1 in Kubo as a thin wrapper around Kubo's ow n routing system, which does not return self in results. Including it makes sense.

@hacdias you may remember the details, how involved it would be to check local blockstore in parallel to kubo routing system, and inject self if we have the block?

ps. NoFetch does not apply to routing info and /routing/v1 results, it only applies to data (fetch of new blocks can't be triggered via Gateway).

[aschmahmann]

This is because /routing/v1 in Kubo as a thin wrapper around Kubo's ow n routing system, which does not return self in results. Including it makes sense.

🤷, I can see either way here. Sure, the node can do a check like blockstore.Has(cid) and append that result to the output of the delegated routing call but the result is likely more confusion. Are there any use cases for this outside of a testing environment?

I can see a few potential issues from doing this:

1. Confusion due to inconsistency
   • If the local routing-v1 returns different results from ipfs routing that could confuse people
     • If the inconsistency with ipfs routing is resolved by having it also return itself then how will users debug if they've advertised their data or not?
   • If the results of routing-v1 requests from different peers with the same config returns different values that could cause some confusion around if data is advertised or not
2. If/when we get something like this IPIP-0388: Routing HTTP API Support for Querying Multiple Routers specs#388 the "I have stuff, but it's in no routing system" wouldn't be found unless you made up a new routing system name like ask-me or just asked for everything without enumerating the routers

That being said if others really feel this should be here I won't object (although if it causes more work around ipfs/specs#388 I'll advocate for dropping this change).

[achingbrain]

Something else to consider here maybe is that if the goal is to have a PeerInfo for the current node appear in it's own routing output if that node has the block being searched for, and that PeerInfo is to contain a HTTP multiaddr, we should be able to override the host/port in that multiaddr.

I.e. browsers will expect a DNS+HTTPS address and will error if it ends up being IP+HTTP.

[aschmahmann]

we should be able to override the host/port in that multiaddr.
...browsers will expect a DNS+HTTPS address and will error if it ends up being IP+HTTP.

Why does the latter imply the former? You can advertise DNS-based multiaddrs and you shouldn't need anything special in the local routing-v1 implementation to do so since you'd want other node's routing-v1 implementations and local queries to also find your DNS address.

If there's some kubo issue with over-resolution of DNS addresses that seems like a separate bug to file and fix.

[gammazero]

@SgtPooki Is this still an issue. If no, please close.

[SgtPooki]

sorry for the delay, looking now

[SgtPooki]

@gammazero It is not fixed, where fixed means the local kubo node returns itself as a provider.

The local kubo node is still not in the list of providers. I updated the output in the reproduction repo: https://github.com/SgtPooki/repro-kubo-not-announcing-self-as-provider/tree/56a5b53868875496524f5922da1cdf8b50712456

Some questions

1. dht providers is empty now? I ran the code before I made any updates of the deps, and dht providers was returning multiple providers. Is this expected?
2. are there any config updates I need to make to see the fix?

[lidel]

We should have a configuration option to control:

• ExposeRoutingAPI if external routing systems should be queried (keep current opt-in behavior)
• ExposeOfflineRoutingAPI if local datastore is checked, and if block is in local blockstore, local peerid is returned as one of providers (new, enable by default, allow opt-out)

Those two settings will allow Kubo to expose /routing/v1 endpoint that reflects local state.

Together with ipfs-cluster (having multiple Kubo's like that) and caching delegated routing proxies like someguy (querying those kubo's /routing/v1 endpoints) this enables people to self-host big amounts of data without announcing them on DHT, and provide delegated routing endpoint that can be plugged into existing systems.

Wider context:

• Native HTTP across the IPFS Stack, to enable Filecoin direct retrieval — IPFS/2025 ipshipyard/roadmaps#9 (the "native HTTP" part of it)
• Reduce the amount of spamming (aka network noise) of non-LAN nodes in kubo’s downloading system — IPFS/2025 ipshipyard/roadmaps#10 (if HTTP routing can be used, and Bitswap server can be disabled – config: Flag to disable Bitswap client and server #10717)
• Kubo users can more directly control which data is advertised — IPFS/2025 ipshipyard/roadmaps#8 (as there is no good option for people who dont want to use DHT/IPNI, but still want to "provide" data to be queried/fetched).
• Potential alternative/complimentary feature is SOMEGUY_HTTP_BLOCK_PROVIDER_ENDPOINTS as routing proxy for trustless gateways someguy#109