Skip to content

IPIP-373: Double Hash DHT Spec #373

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 58 commits into
base: main
Choose a base branch
from
Open
Changes from 2 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
91bc1a8
draft double-hash-dht IPIP
guillaumemichel Jan 20, 2023
e1ed413
fixing Github markdown
guillaumemichel Jan 23, 2023
ad5b1ac
added cryptographic algorithms rationale
guillaumemichel Jan 23, 2023
ead1e1d
completed benefits section
guillaumemichel Jan 30, 2023
89cb1ed
added threat model section
guillaumemichel Jan 31, 2023
0a138da
quick spell checks
guillaumemichel Jan 31, 2023
942a1f4
added summary
guillaumemichel Jan 31, 2023
40b44d4
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 1, 2023
6c260f0
update after ischasny comments
guillaumemichel Feb 1, 2023
ac76a25
Merge branch 'double-hashing-dht' of https://github.com/guillaumemich…
guillaumemichel Feb 1, 2023
2c20b47
replaced with for the DHT Server encrypted payload response
guillaumemichel Feb 1, 2023
591fdef
added MatchLimit explanations
guillaumemichel Feb 8, 2023
6dafa77
added aes-256 as varint for aesgcm
guillaumemichel Feb 8, 2023
28383a2
define Provider Records life duration
guillaumemichel Feb 8, 2023
9e8e640
added more specific data formats
guillaumemichel Feb 8, 2023
dbad7fd
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
576be8b
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
941f30a
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
cac3d40
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
c046b7b
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
df3d039
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
5556519
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
b7fca73
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
cd225d5
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
1b189df
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
c8dc975
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
c118325
addressed reviews
guillaumemichel Feb 14, 2023
2a576b6
Merge branch 'double-hashing-dht' of https://github.com/guillaumemich…
guillaumemichel Feb 14, 2023
422f1d5
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 14, 2023
fa0c5a4
addressed reviews
guillaumemichel Feb 14, 2023
6c58794
restructured provider store section
guillaumemichel Feb 14, 2023
9936e75
added open question about multiple matching Provider Records indexing
guillaumemichel Feb 14, 2023
145869b
rephrased open question on using timestamp as IV
guillaumemichel Feb 14, 2023
dcfbcb5
corrected aes gcm varint
guillaumemichel Feb 14, 2023
6cb0f2a
added varint to encryption with ServerKey
guillaumemichel Feb 14, 2023
dca8991
Update IPIP/0000-double-hash-dht.md
guillaumemichel Feb 15, 2023
ba2f3c7
added mermaid diagrams
guillaumemichel Feb 15, 2023
aba292b
fix copy paste
guillaumemichel Feb 15, 2023
4251903
reverting template
guillaumemichel Feb 15, 2023
b1ad802
correcting modified template
guillaumemichel Feb 15, 2023
c2aabcb
Removed a modified file from pull request
guillaumemichel Feb 15, 2023
3574890
updated aes-gcm varint
guillaumemichel Feb 27, 2023
c575685
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
c188425
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
6356f58
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
60fdd80
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
103ebcd
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
848c2a9
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
a9caf14
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
436d066
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
11525c5
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
37aa63d
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
67a3dfd
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
5325d55
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
0da9010
Update IPIP/0373-double-hash-dht.md
guillaumemichel Feb 28, 2023
08aafde
modifying enum numbers in algorithm
guillaumemichel Mar 1, 2023
cbaf930
merged commits
guillaumemichel Mar 1, 2023
79a915f
data format update
guillaumemichel Mar 1, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions IPIP/0373-double-hash-dht.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,6 @@ The following process describes the event of a client looking up a CID in the IP
9. Go to 4.
10. The DHT servers storing the Provider Record(s) associated with `MH` send them to Client. (Currently, if a Provider Record has been published less than 30 min before being requested, the DHT servers also send the `multiaddresses` of the Content Provider to Client).
11. If the response from the DHT server doesn't include the `multiaddrs` associated with the Content Providers' `PeerID`s, Client performs a DHT `FindPeer` request to find the `multiaddrs` of the returned `PeerID`s.
12. Client sends a Bitswap request for `CID` to the Content Provider (known `PeerID` and `multiaddrs`).
13. Content Provider sends the requested content back to Client.

### Double Hash DHT design

Expand Down Expand Up @@ -126,6 +124,25 @@ sequenceDiagram
**Lookup Process**
1. Client computes `HASH2 = SHA256(SALT_DOUBLEHASH || MH)` (`MH` is the MultiHash included in the CID).
2. Client selects a prefix of `HASH2`, `KeyPrefix = HASH2[:l]` for a defined `l` (see [`l` selection](#prefix-length-selection)).
<<<<<<< HEAD
3. Client finds the closest `PeerID`s to `HASH2` in XOR distance in its Routing Table.
4. Client sends a DHT lookup request for `KeyPrefix` to these DHT servers. The request contains a flag to specify whether Client wants the `multiaddrs` associated with the `CPPeerID` or not. <!-- TODO: add multiaddrs request flag -->
5. The DHT servers find the 20 closest `PeerID`s to `KeyPrefix` in XOR distance (see [algorithm](#closest-keys-to-a-key-prefix)). Add these `PeerID`s and their associated multiaddresses (if applicable) to the `message` that will be returned to Client.
6. The DHT servers search if there are entries matching `KeyPrefix` in their Provider Store.
7. For all entries `HASH2` of the Provider Store where `HASH2[:len(KeyPrefix)]==KeyPrefix`, add to `message` the following encrypted payload: `EncPeerID || 0x8040 || SERVERNONCE || payload_len || AESGCM(ServerKey, SERVERNONCE, TS || Signature || multiaddrs)`, `SERVERNONCE` being a randomly generated 12-byte array, for `multiaddrs` being the multiaddresses associated with `CPPeerID` (if applicable) if the `multiaddrs` were requested by Client. The `multiaddrs` are taken from the DHT Server's lib2p2 peerstore, and may be stale. If more than `MatchLimit` distinct `HASH2`s match the requested `KeyPrefix`, the DHT Server doesn't return any Provider Record, and adds the number of `HASH2` matching `KeyPrefix` along with its own `MatchLimit` to `message`.
8. The DHT servers send `message` to Client.
9. Client computes `ServerKey = SHA256(SALT_SERVERKEY || MH)`.
10. Client tries to decrypt all returned encrypted payloads using `MH` for `EncPeerID` and `ServerKey` for `Enc(ServerKey, TS || Signature || multiaddrs)`. If at least one encrypted payload can be decrypted, go to 13.
11. If the DHT Server's `MatchLimit` and number of matching `HASH2`s was included in the `message`, Client makes multiple DHT lookup requests for longer prefixes (e.g `KeyPrefix||0` and `KeyPrefix||1`). Else Client sends a DHT lookup request for `KeyPrefix` to the closest peers in XOR distance to `HASH2` that it received from the DHT servers.
12. Go to 5.
13. For each decrypted payload, Client decrypts `CPPeerID = Dec(MH, EncPeerID)`.
14. Client verifies that `Signature` verifies with `CPPeerID`: `Verify(CPPeerID, Signature, EncPeerID || TS)`.
15. Client checks that `TS` is younger than `48h`.
16. If none of the decrypted payloads is valid, go to 5.
17. If the decrypted payload doesn't include the `multiaddrs` associated with `CPPeerID`, Client performs a DHT `FindPeer` request to find the `multiaddrs` associated with `CPPeerID`.
18. Client requests `CID` or another content identifier to the Content Provider (known `CPPeerID` and `multiaddrs`) and can exchange data (the DHT may be consumed by various different protocols).

=======
2. Client finds the closest `PeerID`s to `HASH2` in XOR distance in its Routing Table.
3. Client sends a DHT lookup request for `KeyPrefix` to these DHT servers. The request contains a flag to specify whether Client wants the `multiaddrs` associated with the `CPPeerID` or not. <!-- TODO: add multiaddrs request flag -->
4. The DHT servers find the 20 closest `PeerID`s to `KeyPrefix` in XOR distance (see [algorithm](#closest-keys-to-a-key-prefix)). Add these `PeerID`s and their associated multiaddresses (if applicable) to the `message` that will be returned to Client.
Expand All @@ -143,6 +160,7 @@ sequenceDiagram
16. If the decrypted payload doesn't include the `multiaddrs` associated with `CPPeerID`, Client performs a DHT `FindPeer` request to find the `multiaddrs` associated with `CPPeerID`.
17. Client sends a Bitswap request for `CID` to the Content Provider (known `CPPeerID` and `multiaddrs`).
18. Content Provider sends the requested content back to Client.
>>>>>>> 0da90106a2ac73045f1b6d033ac8a6f486e243e0

```mermaid
sequenceDiagram
Expand Down