Skip to content

Add multi-worker work support to BeaconProcessor #7720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 1 commit into
base: unstable
Choose a base branch
from
Draft

Add multi-worker work support to BeaconProcessor #7720

wants to merge 1 commit into from
+192 −2

Conversation

jimmygchen
Copy link
Member

@jimmygchen jimmygchen commented Jul 9, 2025 •
edited
Loading

Issue Addressed

Proposed Changes

(Don't look, the code it splits out is not great 😅 )

Implements generic multi-worker configuration allowing tasks to allocate multiple workers with scoped rayon thread pools:

  • Per-work-type configuration: Each work type can specify worker count (defaults to 1)
  • Column reconstruction: Uses min(4, max_workers) by default
  • Scoped thread pools: Prevents oversubscription by coordinating BeaconProcessor and rayon
  • Implement spawn_multi_worker() with scoped rayon thread pools

NOTE: This hasn't been reviewed / tested. Please do NOT merge.

@jimmygchen jimmygchen added do-not-merge optimization Something to make Lighthouse run more efficiently. labels Jul 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
do-not-merge optimization Something to make Lighthouse run more efficiently.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@jimmygchen