Skip to content

feat: sort packed attestations by effective balance #7646

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Apr 2, 2025

Motivation

  • right now we sort packed attestations by attester count which is not correct for electra
  • the not seen attester count is not correct, it should be reevaluated after the previous attestation is included. This is to prevent useless attestations
  • no metrics when producing packed attestations

Description

  • track packed attestations by not seen effective balance
  • reevaluate every attestation after an attestation is included, see getMostValuableAttestation(). This means there is no useless attestations included
  • add a lot of metrics:
    • committee bits for each packed attestation
    • total committee members of each packed attestation
    • non-participation of each packed attestation
    • distance from block slot to attestation slot for each packed attestation
    • total effective balance for each packed attestations
    • scanned slots and termination reason
    • total slots in pool at the time of producing block
    • total consolidations

Closes #7544

TODOs

  • more unit tests for electra
  • add benchmark
  • monitor on hoodi

@twoeths
Copy link
Contributor Author

twoeths commented Apr 2, 2025

some metrics monitor on hoodie

Screenshot 2025-04-02 at 16 06 14
  • effective balance of included attestations
Screenshot 2025-04-02 at 16 07 47
  • scanned slots (+ termination reason) and total slots: it shows that we mostly have to scan all slots and not have up to 8 attestations
Screenshot 2025-04-02 at 16 08 47
  • inclusion distance for each attestation
Screenshot 2025-04-02 at 16 09 18
  • non participation rate
Screenshot 2025-04-02 at 16 10 06 Screenshot 2025-04-02 at 16 10 56

Copy link
Contributor

github-actions bot commented Apr 2, 2025

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 1a088aa Previous: 091881a Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 999.31 us/op 957.29 us/op 1.04
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 36.218 us/op 36.454 us/op 0.99
BLS verify - blst 847.25 us/op 841.66 us/op 1.01
BLS verifyMultipleSignatures 3 - blst 1.1996 ms/op 1.2052 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst 1.6479 ms/op 1.6740 ms/op 0.98
BLS verifyMultipleSignatures 32 - blst 4.8586 ms/op 4.9921 ms/op 0.97
BLS verifyMultipleSignatures 64 - blst 8.9597 ms/op 9.2057 ms/op 0.97
BLS verifyMultipleSignatures 128 - blst 17.523 ms/op 17.154 ms/op 1.02
BLS deserializing 10000 signatures 690.01 ms/op 702.53 ms/op 0.98
BLS deserializing 100000 signatures 6.9416 s/op 7.0979 s/op 0.98
BLS verifyMultipleSignatures - same message - 3 - blst 936.90 us/op 976.30 us/op 0.96
BLS verifyMultipleSignatures - same message - 8 - blst 1.0629 ms/op 1.0988 ms/op 0.97
BLS verifyMultipleSignatures - same message - 32 - blst 1.7809 ms/op 1.7431 ms/op 1.02
BLS verifyMultipleSignatures - same message - 64 - blst 2.6463 ms/op 2.7075 ms/op 0.98
BLS verifyMultipleSignatures - same message - 128 - blst 4.3919 ms/op 4.4913 ms/op 0.98
BLS aggregatePubkeys 32 - blst 19.751 us/op 20.225 us/op 0.98
BLS aggregatePubkeys 128 - blst 70.767 us/op 72.302 us/op 0.98
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 51.251 ms/op 49.624 ms/op 1.03
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 52.211 ms/op 50.953 ms/op 1.02
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 39.740 ms/op 39.280 ms/op 1.01
getSlashingsAndExits - default max 74.570 us/op 82.508 us/op 0.90
getSlashingsAndExits - 2k 333.86 us/op 389.25 us/op 0.86
proposeBlockBody type=full, size=empty 5.7776 ms/op 8.4558 ms/op 0.68
isKnown best case - 1 super set check 210.00 ns/op 217.00 ns/op 0.97
isKnown normal case - 2 super set checks 205.00 ns/op 211.00 ns/op 0.97
isKnown worse case - 16 super set checks 205.00 ns/op 209.00 ns/op 0.98
InMemoryCheckpointStateCache - add get delete 2.6020 us/op 2.7290 us/op 0.95
validate api signedAggregateAndProof - struct 1.5890 ms/op 1.7808 ms/op 0.89
validate gossip signedAggregateAndProof - struct 1.4312 ms/op 1.6375 ms/op 0.87
batch validate gossip attestation - vc 640000 - chunk 32 121.27 us/op 127.65 us/op 0.95
batch validate gossip attestation - vc 640000 - chunk 64 109.13 us/op 110.26 us/op 0.99
batch validate gossip attestation - vc 640000 - chunk 128 106.21 us/op 106.44 us/op 1.00
batch validate gossip attestation - vc 640000 - chunk 256 110.71 us/op 107.92 us/op 1.03
pickEth1Vote - no votes 997.97 us/op 994.55 us/op 1.00
pickEth1Vote - max votes 7.8657 ms/op 7.5792 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.796 ms/op 13.659 ms/op 0.94
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.671 ms/op 18.943 ms/op 1.09
pickEth1Vote - Eth1Data fastSerialize value x2048 458.25 us/op 424.31 us/op 1.08
pickEth1Vote - Eth1Data fastSerialize tree x2048 2.6467 ms/op 2.1622 ms/op 1.22
bytes32 toHexString 391.00 ns/op 371.00 ns/op 1.05
bytes32 Buffer.toString(hex) 253.00 ns/op 232.00 ns/op 1.09
bytes32 Buffer.toString(hex) from Uint8Array 357.00 ns/op 335.00 ns/op 1.07
bytes32 Buffer.toString(hex) + 0x 252.00 ns/op 235.00 ns/op 1.07
Object access 1 prop 0.12600 ns/op 0.11700 ns/op 1.08
Map access 1 prop 0.13300 ns/op 0.12400 ns/op 1.07
Object get x1000 6.3800 ns/op 6.0180 ns/op 1.06
Map get x1000 7.1740 ns/op 6.6160 ns/op 1.08
Object set x1000 32.972 ns/op 31.266 ns/op 1.05
Map set x1000 22.334 ns/op 21.106 ns/op 1.06
Return object 10000 times 0.31530 ns/op 0.29650 ns/op 1.06
Throw Error 10000 times 4.7624 us/op 4.5953 us/op 1.04
toHex 141.97 ns/op 144.74 ns/op 0.98
Buffer.from 124.43 ns/op 122.07 ns/op 1.02
shared Buffer 81.001 ns/op 79.247 ns/op 1.02
fastMsgIdFn sha256 / 200 bytes 2.2930 us/op 2.2470 us/op 1.02
fastMsgIdFn h32 xxhash / 200 bytes 211.00 ns/op 216.00 ns/op 0.98
fastMsgIdFn h64 xxhash / 200 bytes 284.00 ns/op 275.00 ns/op 1.03
fastMsgIdFn sha256 / 1000 bytes 7.7000 us/op 7.4330 us/op 1.04
fastMsgIdFn h32 xxhash / 1000 bytes 349.00 ns/op 344.00 ns/op 1.01
fastMsgIdFn h64 xxhash / 1000 bytes 361.00 ns/op 352.00 ns/op 1.03
fastMsgIdFn sha256 / 10000 bytes 67.816 us/op 70.273 us/op 0.97
fastMsgIdFn h32 xxhash / 10000 bytes 1.9110 us/op 1.9280 us/op 0.99
fastMsgIdFn h64 xxhash / 10000 bytes 1.2660 us/op 1.3030 us/op 0.97
send data - 1000 256B messages 16.859 ms/op 15.641 ms/op 1.08
send data - 1000 512B messages 23.133 ms/op 16.853 ms/op 1.37
send data - 1000 1024B messages 26.484 ms/op 29.726 ms/op 0.89
send data - 1000 1200B messages 27.791 ms/op 21.851 ms/op 1.27
send data - 1000 2048B messages 30.522 ms/op 27.667 ms/op 1.10
send data - 1000 4096B messages 33.374 ms/op 26.587 ms/op 1.26
send data - 1000 16384B messages 64.931 ms/op 67.479 ms/op 0.96
send data - 1000 65536B messages 279.86 ms/op 239.14 ms/op 1.17
enrSubnets - fastDeserialize 64 bits 1.9650 us/op 903.00 ns/op 2.18
enrSubnets - ssz BitVector 64 bits 342.00 ns/op 333.00 ns/op 1.03
enrSubnets - fastDeserialize 4 bits 140.00 ns/op 130.00 ns/op 1.08
enrSubnets - ssz BitVector 4 bits 341.00 ns/op 331.00 ns/op 1.03
prioritizePeers score -10:0 att 32-0.1 sync 2-0 127.24 us/op 119.71 us/op 1.06
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 149.40 us/op 145.76 us/op 1.02
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 216.17 us/op 209.31 us/op 1.03
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 406.04 us/op 390.18 us/op 1.04
prioritizePeers score 0:0 att 64-1 sync 4-1 488.12 us/op 471.43 us/op 1.04
array of 16000 items push then shift 1.6926 us/op 1.6829 us/op 1.01
LinkedList of 16000 items push then shift 9.1350 ns/op 7.7210 ns/op 1.18
array of 16000 items push then pop 86.659 ns/op 81.428 ns/op 1.06
LinkedList of 16000 items push then pop 9.4880 ns/op 7.3090 ns/op 1.30
array of 24000 items push then shift 2.5424 us/op 2.5269 us/op 1.01
LinkedList of 24000 items push then shift 9.7200 ns/op 7.6530 ns/op 1.27
array of 24000 items push then pop 120.03 ns/op 115.14 ns/op 1.04
LinkedList of 24000 items push then pop 8.5850 ns/op 7.4090 ns/op 1.16
intersect bitArray bitLen 8 6.8690 ns/op 6.5010 ns/op 1.06
intersect array and set length 8 41.654 ns/op 38.901 ns/op 1.07
intersect bitArray bitLen 128 32.268 ns/op 30.448 ns/op 1.06
intersect array and set length 128 645.21 ns/op 637.69 ns/op 1.01
bitArray.getTrueBitIndexes() bitLen 128 1.1400 us/op 1.0620 us/op 1.07
bitArray.getTrueBitIndexes() bitLen 248 1.9640 us/op 1.8650 us/op 1.05
bitArray.getTrueBitIndexes() bitLen 512 4.1530 us/op 3.8280 us/op 1.08
Buffer.concat 32 items 666.00 ns/op 622.00 ns/op 1.07
Uint8Array.set 32 items 1.7220 us/op 1.2780 us/op 1.35
Buffer.copy 2.8270 us/op 2.1270 us/op 1.33
Uint8Array.set - with subarray 2.8570 us/op 1.6170 us/op 1.77
Uint8Array.set - without subarray 1.4920 us/op 1.4680 us/op 1.02
getUint32 - dataview 205.00 ns/op 197.00 ns/op 1.04
getUint32 - manual 130.00 ns/op 120.00 ns/op 1.08
Set add up to 64 items then delete first 3.0593 us/op 2.4574 us/op 1.24
OrderedSet add up to 64 items then delete first 5.2211 us/op 3.8418 us/op 1.36
Set add up to 64 items then delete last 3.4545 us/op 2.5982 us/op 1.33
OrderedSet add up to 64 items then delete last 5.0927 us/op 4.4738 us/op 1.14
Set add up to 64 items then delete middle 3.4330 us/op 2.8259 us/op 1.21
OrderedSet add up to 64 items then delete middle 7.1027 us/op 5.6177 us/op 1.26
Set add up to 128 items then delete first 7.1002 us/op 5.9150 us/op 1.20
OrderedSet add up to 128 items then delete first 12.044 us/op 9.6162 us/op 1.25
Set add up to 128 items then delete last 6.7009 us/op 5.6460 us/op 1.19
OrderedSet add up to 128 items then delete last 10.592 us/op 8.5819 us/op 1.23
Set add up to 128 items then delete middle 5.9935 us/op 6.1306 us/op 0.98
OrderedSet add up to 128 items then delete middle 17.602 us/op 15.376 us/op 1.14
Set add up to 256 items then delete first 14.119 us/op 11.086 us/op 1.27
OrderedSet add up to 256 items then delete first 22.003 us/op 19.476 us/op 1.13
Set add up to 256 items then delete last 13.242 us/op 10.198 us/op 1.30
OrderedSet add up to 256 items then delete last 21.954 us/op 15.410 us/op 1.42
Set add up to 256 items then delete middle 13.540 us/op 10.054 us/op 1.35
OrderedSet add up to 256 items then delete middle 50.118 us/op 46.581 us/op 1.08
transfer serialized Status (84 B) 2.5110 us/op 2.4740 us/op 1.01
copy serialized Status (84 B) 1.3970 us/op 1.2960 us/op 1.08
transfer serialized SignedVoluntaryExit (112 B) 2.5600 us/op 2.4680 us/op 1.04
copy serialized SignedVoluntaryExit (112 B) 1.5390 us/op 1.3730 us/op 1.12
transfer serialized ProposerSlashing (416 B) 3.1090 us/op 2.4630 us/op 1.26
copy serialized ProposerSlashing (416 B) 2.1230 us/op 1.4820 us/op 1.43
transfer serialized Attestation (485 B) 3.1200 us/op 2.5510 us/op 1.22
copy serialized Attestation (485 B) 1.8230 us/op 1.4100 us/op 1.29
transfer serialized AttesterSlashing (33232 B) 3.6100 us/op 2.6040 us/op 1.39
copy serialized AttesterSlashing (33232 B) 5.8110 us/op 4.4420 us/op 1.31
transfer serialized Small SignedBeaconBlock (128000 B) 4.0980 us/op 3.4640 us/op 1.18
copy serialized Small SignedBeaconBlock (128000 B) 16.335 us/op 15.385 us/op 1.06
transfer serialized Avg SignedBeaconBlock (200000 B) 4.6900 us/op 3.7920 us/op 1.24
copy serialized Avg SignedBeaconBlock (200000 B) 24.704 us/op 16.864 us/op 1.46
transfer serialized BlobsSidecar (524380 B) 5.5290 us/op 4.1820 us/op 1.32
copy serialized BlobsSidecar (524380 B) 88.060 us/op 68.666 us/op 1.28
transfer serialized Big SignedBeaconBlock (1000000 B) 5.6980 us/op 4.6910 us/op 1.21
copy serialized Big SignedBeaconBlock (1000000 B) 190.38 us/op 122.08 us/op 1.56
pass gossip attestations to forkchoice per slot 2.9322 ms/op 2.8957 ms/op 1.01
forkChoice updateHead vc 100000 bc 64 eq 0 479.10 us/op 476.71 us/op 1.01
forkChoice updateHead vc 600000 bc 64 eq 0 3.6943 ms/op 3.3831 ms/op 1.09
forkChoice updateHead vc 1000000 bc 64 eq 0 6.2823 ms/op 5.8801 ms/op 1.07
forkChoice updateHead vc 600000 bc 320 eq 0 4.1426 ms/op 3.0961 ms/op 1.34
forkChoice updateHead vc 600000 bc 1200 eq 0 3.6483 ms/op 3.0430 ms/op 1.20
forkChoice updateHead vc 600000 bc 7200 eq 0 4.0991 ms/op 3.5371 ms/op 1.16
forkChoice updateHead vc 600000 bc 64 eq 1000 11.121 ms/op 10.692 ms/op 1.04
forkChoice updateHead vc 600000 bc 64 eq 10000 11.054 ms/op 11.474 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 300000 26.887 ms/op 14.435 ms/op 1.86
computeDeltas 500000 validators 300 proto nodes 4.8872 ms/op 4.1459 ms/op 1.18
computeDeltas 500000 validators 1200 proto nodes 5.2287 ms/op 4.1241 ms/op 1.27
computeDeltas 500000 validators 7200 proto nodes 4.6171 ms/op 4.0935 ms/op 1.13
computeDeltas 750000 validators 300 proto nodes 6.7423 ms/op 6.0292 ms/op 1.12
computeDeltas 750000 validators 1200 proto nodes 6.6020 ms/op 5.9793 ms/op 1.10
computeDeltas 750000 validators 7200 proto nodes 6.4743 ms/op 6.0338 ms/op 1.07
computeDeltas 1400000 validators 300 proto nodes 13.048 ms/op 11.181 ms/op 1.17
computeDeltas 1400000 validators 1200 proto nodes 13.595 ms/op 11.152 ms/op 1.22
computeDeltas 1400000 validators 7200 proto nodes 11.314 ms/op 11.103 ms/op 1.02
computeDeltas 2100000 validators 300 proto nodes 16.496 ms/op 16.865 ms/op 0.98
computeDeltas 2100000 validators 1200 proto nodes 16.454 ms/op 17.282 ms/op 0.95
computeDeltas 2100000 validators 7200 proto nodes 16.447 ms/op 17.098 ms/op 0.96
altair processAttestation - 250000 vs - 7PWei normalcase 2.0815 ms/op 2.5986 ms/op 0.80
altair processAttestation - 250000 vs - 7PWei worstcase 3.0130 ms/op 3.7851 ms/op 0.80
altair processAttestation - setStatus - 1/6 committees join 129.20 us/op 137.80 us/op 0.94
altair processAttestation - setStatus - 1/3 committees join 247.81 us/op 260.73 us/op 0.95
altair processAttestation - setStatus - 1/2 committees join 348.80 us/op 369.63 us/op 0.94
altair processAttestation - setStatus - 2/3 committees join 434.63 us/op 467.96 us/op 0.93
altair processAttestation - setStatus - 4/5 committees join 616.32 us/op 665.95 us/op 0.93
altair processAttestation - setStatus - 100% committees join 874.14 us/op 766.60 us/op 1.14
altair processBlock - 250000 vs - 7PWei normalcase 11.338 ms/op 4.9063 ms/op 2.31
altair processBlock - 250000 vs - 7PWei normalcase hashState 27.239 ms/op 32.112 ms/op 0.85
altair processBlock - 250000 vs - 7PWei worstcase 38.045 ms/op 38.816 ms/op 0.98
altair processBlock - 250000 vs - 7PWei worstcase hashState 81.913 ms/op 74.873 ms/op 1.09
phase0 processBlock - 250000 vs - 7PWei normalcase 1.7013 ms/op 1.6009 ms/op 1.06
phase0 processBlock - 250000 vs - 7PWei worstcase 23.043 ms/op 25.997 ms/op 0.89
altair processEth1Data - 250000 vs - 7PWei normalcase 357.57 us/op 346.02 us/op 1.03
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.0530 us/op 6.3350 us/op 0.96
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 39.899 us/op 42.967 us/op 0.93
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 9.8520 us/op 11.834 us/op 0.83
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.2940 us/op 7.4710 us/op 0.98
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 143.11 us/op 148.92 us/op 0.96
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0408 ms/op 1.0131 ms/op 1.03
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4358 ms/op 1.4295 ms/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4476 ms/op 1.4275 ms/op 1.01
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.7412 ms/op 3.9917 ms/op 0.94
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.8255 ms/op 1.4455 ms/op 1.26
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.6714 ms/op 3.9626 ms/op 0.93
Tree 40 250000 create 508.32 ms/op 531.58 ms/op 0.96
Tree 40 250000 get(125000) 149.96 ns/op 152.50 ns/op 0.98
Tree 40 250000 set(125000) 1.7135 us/op 1.9074 us/op 0.90
Tree 40 250000 toArray() 22.037 ms/op 22.328 ms/op 0.99
Tree 40 250000 iterate all - toArray() + loop 21.635 ms/op 21.849 ms/op 0.99
Tree 40 250000 iterate all - get(i) 61.840 ms/op 63.895 ms/op 0.97
Array 250000 create 2.5058 ms/op 2.7489 ms/op 0.91
Array 250000 clone - spread 1.5148 ms/op 1.5972 ms/op 0.95
Array 250000 get(125000) 0.40400 ns/op 0.42700 ns/op 0.95
Array 250000 set(125000) 0.42900 ns/op 0.44700 ns/op 0.96
Array 250000 iterate all - loop 83.762 us/op 100.10 us/op 0.84
phase0 afterProcessEpoch - 250000 vs - 7PWei 43.311 ms/op 44.106 ms/op 0.98
Array.fill - length 1000000 3.4889 ms/op 3.6907 ms/op 0.95
Array push - length 1000000 14.483 ms/op 13.216 ms/op 1.10
Array.get 0.27590 ns/op 0.27759 ns/op 0.99
Uint8Array.get 0.44268 ns/op 0.44716 ns/op 0.99
phase0 beforeProcessEpoch - 250000 vs - 7PWei 15.549 ms/op 19.081 ms/op 0.81
altair processEpoch - mainnet_e81889 242.60 ms/op 248.77 ms/op 0.98
mainnet_e81889 - altair beforeProcessEpoch 19.307 ms/op 18.543 ms/op 1.04
mainnet_e81889 - altair processJustificationAndFinalization 5.4660 us/op 5.2980 us/op 1.03
mainnet_e81889 - altair processInactivityUpdates 4.2399 ms/op 4.2120 ms/op 1.01
mainnet_e81889 - altair processRewardsAndPenalties 38.331 ms/op 36.320 ms/op 1.06
mainnet_e81889 - altair processRegistryUpdates 737.00 ns/op 707.00 ns/op 1.04
mainnet_e81889 - altair processSlashings 186.00 ns/op 174.00 ns/op 1.07
mainnet_e81889 - altair processEth1DataReset 183.00 ns/op 170.00 ns/op 1.08
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.3045 ms/op 1.2723 ms/op 1.03
mainnet_e81889 - altair processSlashingsReset 880.00 ns/op 894.00 ns/op 0.98
mainnet_e81889 - altair processRandaoMixesReset 1.2430 us/op 1.1140 us/op 1.12
mainnet_e81889 - altair processHistoricalRootsUpdate 226.00 ns/op 176.00 ns/op 1.28
mainnet_e81889 - altair processParticipationFlagUpdates 554.00 ns/op 520.00 ns/op 1.07
mainnet_e81889 - altair processSyncCommitteeUpdates 151.00 ns/op 140.00 ns/op 1.08
mainnet_e81889 - altair afterProcessEpoch 44.938 ms/op 45.241 ms/op 0.99
capella processEpoch - mainnet_e217614 1.0320 s/op 911.81 ms/op 1.13
mainnet_e217614 - capella beforeProcessEpoch 73.195 ms/op 62.131 ms/op 1.18
mainnet_e217614 - capella processJustificationAndFinalization 5.5670 us/op 5.4030 us/op 1.03
mainnet_e217614 - capella processInactivityUpdates 15.706 ms/op 15.910 ms/op 0.99
mainnet_e217614 - capella processRewardsAndPenalties 171.69 ms/op 176.70 ms/op 0.97
mainnet_e217614 - capella processRegistryUpdates 6.4480 us/op 6.9440 us/op 0.93
mainnet_e217614 - capella processSlashings 190.00 ns/op 173.00 ns/op 1.10
mainnet_e217614 - capella processEth1DataReset 181.00 ns/op 171.00 ns/op 1.06
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.2520 ms/op 5.8457 ms/op 0.73
mainnet_e217614 - capella processSlashingsReset 1.2590 us/op 897.00 ns/op 1.40
mainnet_e217614 - capella processRandaoMixesReset 1.2250 us/op 1.1650 us/op 1.05
mainnet_e217614 - capella processHistoricalRootsUpdate 186.00 ns/op 174.00 ns/op 1.07
mainnet_e217614 - capella processParticipationFlagUpdates 530.00 ns/op 538.00 ns/op 0.99
mainnet_e217614 - capella afterProcessEpoch 118.30 ms/op 117.37 ms/op 1.01
phase0 processEpoch - mainnet_e58758 315.98 ms/op 261.90 ms/op 1.21
mainnet_e58758 - phase0 beforeProcessEpoch 71.824 ms/op 72.731 ms/op 0.99
mainnet_e58758 - phase0 processJustificationAndFinalization 5.6040 us/op 6.0560 us/op 0.93
mainnet_e58758 - phase0 processRewardsAndPenalties 36.403 ms/op 42.830 ms/op 0.85
mainnet_e58758 - phase0 processRegistryUpdates 3.1070 us/op 3.1810 us/op 0.98
mainnet_e58758 - phase0 processSlashings 189.00 ns/op 176.00 ns/op 1.07
mainnet_e58758 - phase0 processEth1DataReset 185.00 ns/op 171.00 ns/op 1.08
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.2121 ms/op 1.1900 ms/op 1.02
mainnet_e58758 - phase0 processSlashingsReset 907.00 ns/op 947.00 ns/op 0.96
mainnet_e58758 - phase0 processRandaoMixesReset 1.3850 us/op 1.1950 us/op 1.16
mainnet_e58758 - phase0 processHistoricalRootsUpdate 226.00 ns/op 176.00 ns/op 1.28
mainnet_e58758 - phase0 processParticipationRecordUpdates 879.00 ns/op 879.00 ns/op 1.00
mainnet_e58758 - phase0 afterProcessEpoch 36.697 ms/op 37.082 ms/op 0.99
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.3267 ms/op 1.5073 ms/op 0.88
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.9328 ms/op 2.0650 ms/op 0.94
altair processInactivityUpdates - 250000 normalcase 18.264 ms/op 17.401 ms/op 1.05
altair processInactivityUpdates - 250000 worstcase 17.145 ms/op 18.492 ms/op 0.93
phase0 processRegistryUpdates - 250000 normalcase 6.7130 us/op 6.9750 us/op 0.96
phase0 processRegistryUpdates - 250000 badcase_full_deposits 298.46 us/op 264.86 us/op 1.13
phase0 processRegistryUpdates - 250000 worstcase 0.5 122.89 ms/op 110.25 ms/op 1.11
altair processRewardsAndPenalties - 250000 normalcase 26.546 ms/op 27.041 ms/op 0.98
altair processRewardsAndPenalties - 250000 worstcase 27.631 ms/op 27.558 ms/op 1.00
phase0 getAttestationDeltas - 250000 normalcase 7.6363 ms/op 7.0272 ms/op 1.09
phase0 getAttestationDeltas - 250000 worstcase 6.0059 ms/op 6.7224 ms/op 0.89
phase0 processSlashings - 250000 worstcase 88.166 us/op 83.763 us/op 1.05
altair processSyncCommitteeUpdates - 250000 11.156 ms/op 11.017 ms/op 1.01
BeaconState.hashTreeRoot - No change 218.00 ns/op 210.00 ns/op 1.04
BeaconState.hashTreeRoot - 1 full validator 75.108 us/op 77.240 us/op 0.97
BeaconState.hashTreeRoot - 32 full validator 905.85 us/op 796.30 us/op 1.14
BeaconState.hashTreeRoot - 512 full validator 9.1807 ms/op 8.9397 ms/op 1.03
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 92.558 us/op 93.759 us/op 0.99
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2705 ms/op 1.3217 ms/op 0.96
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.570 ms/op 18.413 ms/op 1.06
BeaconState.hashTreeRoot - 1 balances 71.949 us/op 74.376 us/op 0.97
BeaconState.hashTreeRoot - 32 balances 664.23 us/op 802.74 us/op 0.83
BeaconState.hashTreeRoot - 512 balances 7.2371 ms/op 7.0026 ms/op 1.03
BeaconState.hashTreeRoot - 250000 balances 159.98 ms/op 140.10 ms/op 1.14
aggregationBits - 2048 els - zipIndexesInBitList 24.015 us/op 21.542 us/op 1.11
byteArrayEquals 32 54.569 ns/op 54.217 ns/op 1.01
Buffer.compare 32 17.722 ns/op 17.114 ns/op 1.04
byteArrayEquals 1024 1.6156 us/op 1.6044 us/op 1.01
Buffer.compare 1024 25.222 ns/op 24.402 ns/op 1.03
byteArrayEquals 16384 25.691 us/op 25.551 us/op 1.01
Buffer.compare 16384 207.36 ns/op 188.02 ns/op 1.10
byteArrayEquals 123687377 195.59 ms/op 192.63 ms/op 1.02
Buffer.compare 123687377 7.8452 ms/op 6.9258 ms/op 1.13
byteArrayEquals 32 - diff last byte 56.766 ns/op 52.639 ns/op 1.08
Buffer.compare 32 - diff last byte 17.625 ns/op 17.092 ns/op 1.03
byteArrayEquals 1024 - diff last byte 1.6288 us/op 1.5917 us/op 1.02
Buffer.compare 1024 - diff last byte 26.458 ns/op 25.758 ns/op 1.03
byteArrayEquals 16384 - diff last byte 25.878 us/op 25.391 us/op 1.02
Buffer.compare 16384 - diff last byte 209.10 ns/op 196.23 ns/op 1.07
byteArrayEquals 123687377 - diff last byte 192.82 ms/op 191.63 ms/op 1.01
Buffer.compare 123687377 - diff last byte 6.7382 ms/op 6.8614 ms/op 0.98
byteArrayEquals 32 - random bytes 5.0850 ns/op 5.1440 ns/op 0.99
Buffer.compare 32 - random bytes 16.943 ns/op 17.194 ns/op 0.99
byteArrayEquals 1024 - random bytes 5.0930 ns/op 5.1090 ns/op 1.00
Buffer.compare 1024 - random bytes 17.321 ns/op 17.142 ns/op 1.01
byteArrayEquals 16384 - random bytes 5.1630 ns/op 5.1180 ns/op 1.01
Buffer.compare 16384 - random bytes 18.188 ns/op 17.256 ns/op 1.05
byteArrayEquals 123687377 - random bytes 6.4400 ns/op 6.3800 ns/op 1.01
Buffer.compare 123687377 - random bytes 18.270 ns/op 18.500 ns/op 0.99
regular array get 100000 times 38.868 us/op 33.138 us/op 1.17
wrappedArray get 100000 times 33.183 us/op 33.080 us/op 1.00
arrayWithProxy get 100000 times 12.820 ms/op 12.674 ms/op 1.01
ssz.Root.equals 46.335 ns/op 46.311 ns/op 1.00
byteArrayEquals 45.389 ns/op 45.408 ns/op 1.00
Buffer.compare 10.311 ns/op 10.306 ns/op 1.00
processSlot - 1 slots 10.220 us/op 10.510 us/op 0.97
processSlot - 32 slots 2.4815 ms/op 2.1166 ms/op 1.17
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 2.9165 ms/op 2.9727 ms/op 0.98
getCommitteeAssignments - req 1 vs - 250000 vc 2.1293 ms/op 2.1111 ms/op 1.01
getCommitteeAssignments - req 100 vs - 250000 vc 4.0316 ms/op 4.1322 ms/op 0.98
getCommitteeAssignments - req 1000 vs - 250000 vc 4.2900 ms/op 4.3710 ms/op 0.98
findModifiedValidators - 10000 modified validators 730.67 ms/op 721.93 ms/op 1.01
findModifiedValidators - 1000 modified validators 714.51 ms/op 698.30 ms/op 1.02
findModifiedValidators - 100 modified validators 226.57 ms/op 213.26 ms/op 1.06
findModifiedValidators - 10 modified validators 182.53 ms/op 152.77 ms/op 1.19
findModifiedValidators - 1 modified validators 170.45 ms/op 246.17 ms/op 0.69
findModifiedValidators - no difference 244.60 ms/op 186.98 ms/op 1.31
compare ViewDUs 6.0425 s/op 6.2367 s/op 0.97
compare each validator Uint8Array 1.4690 s/op 1.4345 s/op 1.02
compare ViewDU to Uint8Array 1.2969 s/op 1.0342 s/op 1.25
migrate state 1000000 validators, 24 modified, 0 new 821.87 ms/op 839.52 ms/op 0.98
migrate state 1000000 validators, 1700 modified, 1000 new 1.1520 s/op 1.0281 s/op 1.12
migrate state 1000000 validators, 3400 modified, 2000 new 1.3127 s/op 1.2209 s/op 1.08
migrate state 1500000 validators, 24 modified, 0 new 781.86 ms/op 796.23 ms/op 0.98
migrate state 1500000 validators, 1700 modified, 1000 new 994.22 ms/op 1.0600 s/op 0.94
migrate state 1500000 validators, 3400 modified, 2000 new 1.2377 s/op 1.2272 s/op 1.01
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.2100 ns/op 4.1700 ns/op 1.01
state getBlockRootAtSlot - 250000 vs - 7PWei 395.80 ns/op 542.41 ns/op 0.73
naive computeProposerIndex 100000 validators 51.506 ms/op 49.752 ms/op 1.04
computeProposerIndex 100000 validators 1.5284 ms/op 1.4914 ms/op 1.02
naiveGetNextSyncCommitteeIndices 1000 validators 8.5478 s/op 7.3443 s/op 1.16
getNextSyncCommitteeIndices 1000 validators 122.90 ms/op 112.12 ms/op 1.10
naiveGetNextSyncCommitteeIndices 10000 validators 8.3541 s/op 7.0932 s/op 1.18
getNextSyncCommitteeIndices 10000 validators 115.27 ms/op 111.53 ms/op 1.03
naiveGetNextSyncCommitteeIndices 100000 validators 6.8117 s/op 7.1288 s/op 0.96
getNextSyncCommitteeIndices 100000 validators 111.82 ms/op 119.29 ms/op 0.94
naive computeShuffledIndex 100000 validators 21.404 s/op 28.606 s/op 0.75
cached computeShuffledIndex 100000 validators 557.12 ms/op 582.52 ms/op 0.96
naive computeShuffledIndex 2000000 validators 600.28 s/op 582.18 s/op 1.03
cached computeShuffledIndex 2000000 validators 51.520 s/op 86.702 s/op 0.59
computeProposers - vc 250000 600.18 us/op 756.67 us/op 0.79
computeEpochShuffling - vc 250000 44.210 ms/op 52.771 ms/op 0.84
getNextSyncCommittee - vc 250000 12.660 ms/op 14.221 ms/op 0.89
computeSigningRoot for AttestationData 24.801 us/op 55.583 us/op 0.45
hash AttestationData serialized data then Buffer.toString(base64) 1.7041 us/op 2.1429 us/op 0.80
toHexString serialized data 1.3219 us/op 2.9979 us/op 0.44
Buffer.toString(base64) 162.82 ns/op 188.81 ns/op 0.86
nodejs block root to RootHex using toHex 142.19 ns/op 171.32 ns/op 0.83
nodejs block root to RootHex using toRootHex 83.225 ns/op 88.522 ns/op 0.94
browser block root to RootHex using the deprecated toHexString 224.53 ns/op 251.09 ns/op 0.89
browser block root to RootHex using toHex 182.52 ns/op 209.41 ns/op 0.87
browser block root to RootHex using toRootHex 163.94 ns/op 175.36 ns/op 0.93

by benchmarkbot/action

Copy link

codecov bot commented Apr 2, 2025

Codecov Report

Attention: Patch coverage is 0% with 76 lines in your changes missing coverage. Please review.

Project coverage is 50.17%. Comparing base (90d2b14) to head (629b0d7).
Report is 1 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #7646      +/-   ##
============================================
- Coverage     50.25%   50.17%   -0.09%     
============================================
  Files           604      604              
  Lines         40479    40549      +70     
  Branches       2231     2230       -1     
============================================
+ Hits          20343    20345       +2     
- Misses        20094    20162      +68     
  Partials         42       42              
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@twoeths
Copy link
Contributor Author

twoeths commented Apr 3, 2025

one issue with this improve aggregated attestation pool is it usually produce too few consolidations/packed attestations (usually 2-7) although it scanned through every slot
need to add a metric to track new seen attesters here and also on chain data to compare

ensi321 pushed a commit that referenced this pull request Apr 5, 2025
**Motivation**

- track on chain new seen attesters and total effective balance
increment of them per block
- see
#7646 (comment)

**Description**

- at every block processing, when process attestations we track it

---------

Co-authored-by: Tuyen Nguyen <[email protected]>
nflaig added a commit that referenced this pull request Apr 7, 2025
**Motivation**

- we have 2 bugs for electra logic:
  - one logic in phase0 cannot be applied to electra
  - wrong for loop place
- one bug for all forks: incorrect dependent root used when checking
attestation data, see #7651

**Description**

extract fixes from #7646
- add more metrics for the pool
- remove this logic for electra: "after 2 slots, there are a good chance
that we have 2 * MAX_ATTESTATIONS_ELECTRA attestations and break the for
loop early"
- correct the for loop place to limit attestation consolidations
- fix `isValidShuffling()` for #7651 
- unit tests are done in #7646 , since logic changes there I cannot
bring them here

---------

Co-authored-by: Tuyen Nguyen <[email protected]>
Co-authored-by: Nico Flaig <[email protected]>
@twoeths
Copy link
Contributor Author

twoeths commented Apr 8, 2025

closing in favor of #7673

@twoeths twoeths closed this Apr 8, 2025
nflaig added a commit that referenced this pull request Apr 11, 2025
**Motivation**

- packed attestations are currently sorted by new seen attesters, should
be by effective balance
- not enough metrics when producing packed attestations

**Description**

- for electra, retain up to 8 attestations per group. When getting
attestations, we also get up to 8 of them (instead of 3)
- track packed attestations by new seen effective balance
- reevaluate every attestation after an attestation is included, see
`getMostValuableAttestation()`. This means there is no useless
attestations included
- add a lot of metrics:
  - committee bits for each packed attestation
  - total committee members of each packed attestation
  - non-participation of each packed attestation
- distance from block slot to attestation slot for each packed
attestation
  - total effective balance for each packed attestations
  - scanned slots and termination reason
  - scanned attestations per scanned slot
  - returned attestations per scanned slot
  - total slots in pool at the time of producing block
  - total consolidations

Closes #7544

**Testing**

- see some metrics here
#7646 (comment)
- performance are the same on hoodi prod node

<img width="846" alt="Screenshot 2025-04-08 at 16 09 17"
src="https://github.com/user-attachments/assets/7ffb3bcb-7774-4383-b2e7-da1065ff8f97"
/>

---------

Co-authored-by: Tuyen Nguyen <[email protected]>
Co-authored-by: Nico Flaig <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add block attestation packing quality metrics
1 participant