deps!: update pkarr to v3 #3186

Frando · 2025-02-14T13:34:44Z

Description

Updates pkarr from v2 to v3. The pkarr client now supports both relays and DHT, so the DHT discovery code becomes a bit simpler.

Breaking Changes

pkarr::SignedPacket, as used as a parameter in iroh::dns::node_info::NodeInfo::to_pkarr_signed_packet and iroh::dns::node_info::NodeInfo::from_pkarr_signed_packet is now expecting pkarr at major version 3 instead of 2

Notes & open questions

Change checklist

Self-review.
Documentation updates following the style guide, if relevant.
Tests if relevant.
All breaking changes documented.

github-actions · 2025-02-14T13:42:42Z

Documentation for this PR has been generated and is available at: https://n0-computer.github.io/iroh/pr/3186/docs/iroh/

Last updated: 2025-04-30T10:14:29Z

github-actions · 2025-02-14T13:49:48Z

Netsim report & logs for this PR have been generated and is available at: LOGS
This report will remain available for 3 days.

Last updated for commit: 435ade7

Nuhvi · 2025-02-14T15:33:29Z

@Frando The wasm error is what I get for updating to rand 0.9.0 which demands getrandom 0.3.0 which requires some massaging, even worse that sometimes you just have to do the 0.2.0 massaging as well if you have other dependencies that require it.

Long story short you need to add getrandom 0.3.0 with wasm_js feature and add this too https://github.com/pubky/pkarr/blob/main/pkarr/.cargo/config.toml

Edit: if you know a better way to do this, let me know, would love to learn.

Nuhvi · 2025-02-14T15:56:17Z

I am not sure how would iroh-relay run in wasm, but anyway, I think the .cargo/config.toml should go in that crate.

Frando · 2025-02-21T09:06:29Z

Note: While all tests pass this is not yet good to go. the PkarrClient in pkarr 3.4 has a bug in that it unconditionally DNS-resolves the default bootstrap URLs even if configured differently (i.e. without DHT or with different bootstrap URLs) because it does the resolving in a Default impl where this is not checked.

even when initializing a ClientBuilderwith no_default_networks or no_dht, the ClientBuilder::default which is called from Client::builder goes to Config::default which unconditionally calls Dht::builder, which at least in the most recent mainline release goes to Dhtbuilder::default and then mainline::rpc::Config::default which unconditionally calls bootstrap: to_socket_address(&DEFAULT_BOOTSTRAP_NODES) which (synchrounsly, blockingly!) DNS-resolves the default bootstrap URLs - even if the DHT is not used because no_default_networks is called
even if using the DHT, a sync-blocking call to to_socket_address for ~4 URLs is a bit of a nogo, it will block the tokio event loop for ~2s for me
but it definitely should not do that if the DHT is disabled in the pkarr client builder
(frando in discord)

So let's wait for a new release that fixes these issues.

Nuhvi · 2025-02-21T12:56:54Z

@Frando published [email protected], and besides fixing the mainline issue, pkarr now re-exports mainline, so you can skip adding mainline as an explicit dependency.

However, Github CI has been failing for me for no reason, even when I rerun older commits that already passed, I seen no issues on mac or linux, so it should be fine, but I am not feeling too much confidence in CIs at the moment, so keep that in mind.

Nuhvi · 2025-03-14T07:40:58Z

pkarr v3.6.0 have few bug fixes, but also enables setting timeout for requests to relays in wasm (after Reqwest published a new version).

Frando · 2025-03-14T10:00:43Z

Alright, I updated this PR:

Merged main
Update to pkarr v3.6
Remove explicit mainline dependency
Remove debug leftovers

Let's see what CI says, but I think this should be good to go now.

Requested a review from @rklaehn as the biggest changes are in the pkarr dht discovery impl. It became quite a bit simpler, as more of the logic to use both relays and the DHT is now handled by pkarr itself.

There's also some changes in the DNS server. To make sure that we don't break anything: @Arqu, can we testdrive this on staging or in a test environment somehow easily?

Nuhvi · 2025-03-14T10:11:05Z

iroh-dns-server/src/lib.rs

@@ -101,13 +101,13 @@ mod tests {
                30,
                dns::rdata::RData::AAAA(Ipv6Addr::LOCALHOST.into()),
            ));
-            SignedPacket::from_packet(&keypair, &packet)?
+            SignedPacket::new(&keypair, &packet.answers, Timestamp::now())?


It is not important here, but if there are other places where you have similar verbosity, reminder that the new SignedPacket::builder() api is much nicer.

Arqu · 2025-03-14T10:19:13Z

Alright, I updated this PR:
* Merged `main`

* Update to `pkarr v3.6`

* Remove explicit mainline dependency

* Remove debug leftovers
Let's see what CI says, but I think this should be good to go now.

Requested a review from @rklaehn as the biggest changes are in the pkarr dht discovery impl. It became quite a bit simpler, as more of the logic to use both relays and the DHT is now handled by pkarr itself.

There's also some changes in the DNS server. To make sure that we don't break anything: @Arqu, can we testdrive this on staging or in a test environment somehow easily?

Yes, we can set this up on the staging dns instance and do some tests.

dignifiedquire · 2025-04-02T09:33:54Z

iroh/src/discovery/pkarr/dht.rs

-                    tracing::info!("discovered node info from DHT {:?}", &node_info);
-                    Some(Ok(DiscoveryItem::new(node_info, "mainline", None)))
+                    tracing::info!("discovered node info {:?}", node_info);
+                    Some(Ok(DiscoveryItem::new(node_info, "dht", None)))


is this always dht or should we rename this to pkarr?

Good point, it supports both relay and DHT, so changed to pkarr.

rklaehn

Changes seem like a straightforward API update.

I tried the dht_discovery example. It still seems to work.

Not sure if pkarr has gotten heavier with this update, but since it is an optional feature I guess we will have to live with it.

Did a quick cargo tree, and didn't see anything unusual. Just the usual rust madness.

… iroh-dns-server v0.34 or lower (#3295) ## Description #3186 introduced a backwards-incompatible change to the iroh-dns-server database that went uncaught so far. The serialization format for SignedPackets changed between pkarr v2 and pkarr v3: pkarr v3 prepends a `last_seen` timestamp (8 bytes) to the serialized representation of a SignedPacket. Therefore, parsing packets stored in the the database with pkarr v2 (iroh-dns-server v0.34 and lower) fails with an error. The error also stops the store actor, and thus all subsequent requests will fail as well. This PR changes this by attempting to parse the packet in the pkarr v2 format if parsing as pkarr v3 packet fails. Parsing as pkarr v2 is achieved by prepending 8 zero bytes to the payload (an empty last_seen timestamp). The PR also improves the tracing logging at various places. ## Breaking Changes  ## Notes & open questions There's no test yet. I tested manually by launching iroh-dns-server v0.34, inserting some packets with the `publish` example. When restarting the server from `main`, retrieving packets leads to errors. When applying this PR, retrieving the old packets work. If we want / need a test, how should we do it? We can either add a dev dependency to `[email protected]` (which would recursively pull in all of `[email protected]`), or somehow insert a packet as it would be created by `[email protected]` manually into the database. Not sure if it's worth it, but maybe it is? ## Change checklist  - [x] Self-review. - [x] Documentation updates following the [style guide](https://rust-lang.github.io/rfcs/1574-more-api-documentation-conventions.html#appendix-a-full-conventions-text), if relevant. - [ ] Tests if relevant.

Frando force-pushed the deps/pkarr-v3 branch from 4bdd95e to 65eefe1 Compare February 14, 2025 13:40

deps: update pkarr to v3

7fac5d9

Frando force-pushed the deps/pkarr-v3 branch from 65eefe1 to 7fac5d9 Compare February 14, 2025 15:02

fix: wasm build

fdc5561

Frando force-pushed the deps/pkarr-v3 branch from 560bb0d to fdc5561 Compare February 14, 2025 15:43

Frando added 3 commits February 14, 2025 17:05

fix: try to make wasm build work

e8fe869

chore: update pkarr in Cargo.lock

ebe373f

Merge remote-tracking branch 'origin/main' into deps/pkarr-v3

d44f3e0

Frando mentioned this pull request Feb 18, 2025

deps: update pkarr to v3 #3188

Closed

4 tasks

fix: pkarr downgraded rand

f94f657

Frando changed the title ~~deps: update pkarr to v3~~ deps!: update pkarr to v3 Feb 19, 2025

Frando added 3 commits February 19, 2025 09:41

chore: trying to get backtraces from cross tests

e75bd10

chore: another try at cross backtrace

0ec16ee

update pkarr patch version

23bc2c6

Frando mentioned this pull request Feb 19, 2025

(ignore) testing pkarr v3 without async compat #3191

Closed

4 tasks

deps: update pkarr version

b6109e3

Frando added 5 commits March 14, 2025 10:52

Merge remote-tracking branch 'origin/main' into deps/pkarr-v3-new

65640d6

deps: update pkarr to 3.6

a912eaf

chore: remove debug leftovers

86ae719

chore: remove cargo update

66ece98

chore: remove explicit mainline dependency, use pkarr reexport instead

d195fd5

Frando marked this pull request as ready for review March 14, 2025 10:01

chore: cleanup

9e3728b

Frando requested a review from rklaehn March 14, 2025 10:10

Nuhvi reviewed Mar 14, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into deps/pkarr-v3

abb4a23

dignifiedquire reviewed Apr 2, 2025

View reviewed changes

Frando mentioned this pull request Apr 2, 2025

tests: make sure the mainline DHT testnet is bootstrapped before using it #3253

Closed

11 tasks

Frando added 3 commits April 2, 2025 22:28

deps: update pkarr to 3.7

8b767ce

change: use new async constructor for Testnet

5e12aff

fix: change provenance label to pkarr

e59fc1b

rklaehn approved these changes Apr 30, 2025

View reviewed changes

rklaehn added 2 commits April 30, 2025 11:59

Merge branch 'main' into deps/pkarr-v3

19d7c57

update cargo.lock from main

aae6c79

rklaehn added this pull request to the merge queue Apr 30, 2025

Merged via the queue into main with commit 7b4bce8 Apr 30, 2025
26 checks passed

github-project-automation bot moved this from 🏗 In progress to ✅ Done in iroh Apr 30, 2025

matheus23 deleted the deps/pkarr-v3 branch April 30, 2025 11:12

Frando mentioned this pull request May 9, 2025

fix(iroh-dns-server): Backwards compatibility for packets stored with iroh-dns-server v0.34 or lower #3295

Merged

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

deps!: update pkarr to v3 #3186

deps!: update pkarr to v3 #3186

Uh oh!

Frando commented Feb 14, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Feb 14, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Feb 14, 2025 •

edited

Loading

Uh oh!

Nuhvi commented Feb 14, 2025 •

edited

Loading

Uh oh!

Nuhvi commented Feb 14, 2025

Uh oh!

Frando commented Feb 21, 2025

Uh oh!

Nuhvi commented Feb 21, 2025 •

edited

Loading

Uh oh!

Nuhvi commented Mar 14, 2025

Uh oh!

Frando commented Mar 14, 2025 •

edited

Loading

Uh oh!

Nuhvi Mar 14, 2025

Uh oh!

Arqu commented Mar 14, 2025

Uh oh!

dignifiedquire Apr 2, 2025

Uh oh!

Frando Apr 2, 2025

Uh oh!

rklaehn left a comment

Uh oh!

Uh oh!

Uh oh!

deps!: update pkarr to v3 #3186

deps!: update pkarr to v3 #3186

Uh oh!

Conversation

Frando commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Breaking Changes

Notes & open questions

Change checklist

Uh oh!

github-actions bot commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nuhvi commented Feb 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nuhvi commented Feb 14, 2025

Uh oh!

Frando commented Feb 21, 2025

Uh oh!

Nuhvi commented Feb 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nuhvi commented Mar 14, 2025

Uh oh!

Frando commented Mar 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Nuhvi Mar 14, 2025

Choose a reason for hiding this comment

Uh oh!

Arqu commented Mar 14, 2025

Uh oh!

dignifiedquire Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

Frando Apr 2, 2025

Choose a reason for hiding this comment

Uh oh!

rklaehn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Frando commented Feb 14, 2025 •

edited

Loading

github-actions bot commented Feb 14, 2025 •

edited

Loading

github-actions bot commented Feb 14, 2025 •

edited

Loading

Nuhvi commented Feb 14, 2025 •

edited

Loading

Nuhvi commented Feb 21, 2025 •

edited

Loading

Frando commented Mar 14, 2025 •

edited

Loading