Skip to content

Make range sync peer loadbalancing PeerDAS-friendly #6922

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 27 commits into from
May 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
fbab829
Remove request tracking inside syncing chains
dapplion Jan 24, 2025
2faf5f1
Prioritize by range peers in network context
dapplion Jan 24, 2025
0e13a8d
Prioritize custody peers for columns by range
dapplion Jan 25, 2025
70e6066
Explicit error handling of the no peers error case
dapplion Jan 25, 2025
8cf4e8c
Remove good_peers_on_sampling_subnets
dapplion Jan 25, 2025
4da322f
Count AwaitingDownload towards the buffer limit
dapplion Jan 25, 2025
9cd238d
Retry syncing chains in AwaitingDownload state
dapplion Jan 25, 2025
891c8fc
Use same peer priorization for lookups
dapplion Feb 5, 2025
29a5aff
Review PR
dapplion Feb 6, 2025
cd6c5d6
Address TODOs
dapplion Feb 6, 2025
2c6a7cc
Revert changes to peer erroring in range sync
dapplion Feb 6, 2025
f77bc24
Revert metrics changes
dapplion Feb 6, 2025
4792275
Update comment
dapplion Feb 6, 2025
45f5528
Pass peers_to_deprioritize to select_columns_by_range_peers_to_request
dapplion Mar 17, 2025
7ec350b
more idiomatic
dapplion Mar 17, 2025
6f7a524
Merge remote-tracking branch 'sigp/unstable' into peer-loadbalancing
dapplion Mar 17, 2025
8962808
Idiomatic while
dapplion Mar 17, 2025
e9247d2
Add note about infinite loop
dapplion Mar 17, 2025
97e75ef
Use while let
dapplion Mar 23, 2025
0e7284f
Merge remote-tracking branch 'sigp/unstable' into peer-loadbalancing
dapplion Mar 25, 2025
5c87e37
Review PR
dapplion Apr 8, 2025
53536c7
Merge remote-tracking branch 'sigp/unstable' into peer-loadbalancing
dapplion Apr 8, 2025
437f54a
Update function doc only
jimmygchen Apr 9, 2025
6567d19
Request columns by range from all synced peers
dapplion May 2, 2025
60f2833
Update doc comment beacon_node/lighthouse_network/src/types/globals.rs
jimmygchen May 5, 2025
658d7a8
Merge branch 'unstable' into peer-loadbalancing
jimmygchen May 5, 2025
c78c568
Add test to verify `request_batches` does not result in infinite loop.
jimmygchen May 5, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions beacon_node/lighthouse_network/src/peer_manager/peerdb.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
use crate::discovery::enr::PEERDAS_CUSTODY_GROUP_COUNT_ENR_KEY;
use crate::discovery::{peer_id_to_node_id, CombinedKey};
use crate::{metrics, multiaddr::Multiaddr, types::Subnet, Enr, EnrExt, Gossipsub, PeerId};
use crate::{
metrics, multiaddr::Multiaddr, types::Subnet, Enr, EnrExt, Gossipsub, PeerId, SyncInfo,
};
use itertools::Itertools;
use logging::crit;
use peer_info::{ConnectionDirection, PeerConnectionStatus, PeerInfo};
Expand All @@ -15,7 +17,7 @@ use std::{
use sync_status::SyncStatus;
use tracing::{debug, error, trace, warn};
use types::data_column_custody_group::compute_subnets_for_node;
use types::{ChainSpec, DataColumnSubnetId, EthSpec};
use types::{ChainSpec, DataColumnSubnetId, Epoch, EthSpec, Hash256, Slot};

pub mod client;
pub mod peer_info;
Expand Down Expand Up @@ -735,6 +737,19 @@ impl<E: EthSpec> PeerDB<E> {
},
);

self.update_sync_status(
&peer_id,
SyncStatus::Synced {
// Fill in mock SyncInfo, only for the peer to return `is_synced() == true`.
info: SyncInfo {
head_slot: Slot::new(0),
head_root: Hash256::ZERO,
finalized_epoch: Epoch::new(0),
finalized_root: Hash256::ZERO,
},
},
);

if supernode {
let peer_info = self.peers.get_mut(&peer_id).expect("peer exists");
let all_subnets = (0..spec.data_column_sidecar_subnet_count)
Expand Down
14 changes: 14 additions & 0 deletions beacon_node/lighthouse_network/src/types/globals.rs
Original file line number Diff line number Diff line change
Expand Up @@ -206,6 +206,20 @@ impl<E: EthSpec> NetworkGlobals<E> {
.collect::<Vec<_>>()
}

/// Returns true if the peer is known and is a custodian of `column_index`
pub fn is_custody_peer_of(&self, column_index: ColumnIndex, peer_id: &PeerId) -> bool {
self.peers
.read()
.peer_info(peer_id)
.map(|info| {
info.is_assigned_to_custody_subnet(&DataColumnSubnetId::from_column_index(
column_index,
&self.spec,
))
})
.unwrap_or(false)
}

/// Returns the TopicConfig to compute the set of Gossip topics for a given fork
pub fn as_topic_config(&self) -> TopicConfig {
TopicConfig {
Expand Down
2 changes: 1 addition & 1 deletion beacon_node/network/src/network_beacon_processor/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1141,7 +1141,7 @@ use {
};

#[cfg(test)]
type TestBeaconChainType<E> =
pub(crate) type TestBeaconChainType<E> =
Witness<ManualSlotClock, CachingEth1Backend<E>, E, MemoryStore<E>, MemoryStore<E>>;

#[cfg(test)]
Expand Down
Loading
Loading