Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add strategy support for state archives #7170

Merged
merged 10 commits into from
Oct 21, 2024

Conversation

nazarhussain
Copy link
Contributor

Motivation

Make the state-archive process more generic so we can extend in future.

Description

  • Make the state-archive process to generic
  • Add feature flag to the cli to select a strategy

Steps to test or reproduce

  • Run all tests

@nazarhussain nazarhussain requested a review from a team as a code owner October 16, 2024 14:15
@nazarhussain nazarhussain self-assigned this Oct 16, 2024
@@ -0,0 +1,163 @@
import {Logger} from "@lodestar/utils";
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is actually old index.ts file, with only strategy specific changes.

Copy link
Contributor

github-actions bot commented Oct 16, 2024

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 0074a97 Previous: b1b5623 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.9436 ms/op 1.7892 ms/op 1.09
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 54.143 us/op 38.001 us/op 1.42
BLS verify - blst 894.17 us/op 897.95 us/op 1.00
BLS verifyMultipleSignatures 3 - blst 1.6412 ms/op 1.3066 ms/op 1.26
BLS verifyMultipleSignatures 8 - blst 1.9708 ms/op 2.1210 ms/op 0.93
BLS verifyMultipleSignatures 32 - blst 5.7194 ms/op 4.4409 ms/op 1.29
BLS verifyMultipleSignatures 64 - blst 11.066 ms/op 8.4126 ms/op 1.32
BLS verifyMultipleSignatures 128 - blst 18.200 ms/op 16.018 ms/op 1.14
BLS deserializing 10000 signatures 705.37 ms/op 620.40 ms/op 1.14
BLS deserializing 100000 signatures 7.2141 s/op 6.1995 s/op 1.16
BLS verifyMultipleSignatures - same message - 3 - blst 1.0155 ms/op 954.87 us/op 1.06
BLS verifyMultipleSignatures - same message - 8 - blst 1.0906 ms/op 1.0434 ms/op 1.05
BLS verifyMultipleSignatures - same message - 32 - blst 1.7455 ms/op 1.6406 ms/op 1.06
BLS verifyMultipleSignatures - same message - 64 - blst 2.6541 ms/op 2.4460 ms/op 1.09
BLS verifyMultipleSignatures - same message - 128 - blst 4.4444 ms/op 3.9680 ms/op 1.12
BLS aggregatePubkeys 32 - blst 20.828 us/op 18.021 us/op 1.16
BLS aggregatePubkeys 128 - blst 72.748 us/op 63.128 us/op 1.15
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 72.373 ms/op 59.430 ms/op 1.22
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 65.425 ms/op 42.433 ms/op 1.54
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 47.337 ms/op 36.511 ms/op 1.30
getSlashingsAndExits - default max 108.51 us/op 71.069 us/op 1.53
getSlashingsAndExits - 2k 322.58 us/op 216.37 us/op 1.49
proposeBlockBody type=full, size=empty 7.1964 ms/op 4.8571 ms/op 1.48
isKnown best case - 1 super set check 389.00 ns/op 464.00 ns/op 0.84
isKnown normal case - 2 super set checks 354.00 ns/op 456.00 ns/op 0.78
isKnown worse case - 16 super set checks 352.00 ns/op 457.00 ns/op 0.77
InMemoryCheckpointStateCache - add get delete 2.8530 us/op 2.6540 us/op 1.07
updateUnfinalizedPubkeys - updating 10 pubkeys 1.6216 ms/op 671.34 us/op 2.42
updateUnfinalizedPubkeys - updating 100 pubkeys 4.1600 ms/op 2.8378 ms/op 1.47
updateUnfinalizedPubkeys - updating 1000 pubkeys 53.722 ms/op 41.781 ms/op 1.29
validate api signedAggregateAndProof - struct 2.0036 ms/op 1.9415 ms/op 1.03
validate gossip signedAggregateAndProof - struct 1.7701 ms/op 1.9275 ms/op 0.92
batch validate gossip attestation - vc 640000 - chunk 32 138.86 us/op 118.83 us/op 1.17
batch validate gossip attestation - vc 640000 - chunk 64 128.00 us/op 105.00 us/op 1.22
batch validate gossip attestation - vc 640000 - chunk 128 112.94 us/op 96.760 us/op 1.17
batch validate gossip attestation - vc 640000 - chunk 256 108.98 us/op 94.103 us/op 1.16
pickEth1Vote - no votes 1.0985 ms/op 897.98 us/op 1.22
pickEth1Vote - max votes 6.2048 ms/op 5.4872 ms/op 1.13
pickEth1Vote - Eth1Data hashTreeRoot value x2048 12.806 ms/op 13.959 ms/op 0.92
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 18.974 ms/op 19.537 ms/op 0.97
pickEth1Vote - Eth1Data fastSerialize value x2048 522.31 us/op 369.82 us/op 1.41
pickEth1Vote - Eth1Data fastSerialize tree x2048 2.8015 ms/op 3.1327 ms/op 0.89
bytes32 toHexString 482.00 ns/op 596.00 ns/op 0.81
bytes32 Buffer.toString(hex) 260.00 ns/op 417.00 ns/op 0.62
bytes32 Buffer.toString(hex) from Uint8Array 445.00 ns/op 521.00 ns/op 0.85
bytes32 Buffer.toString(hex) + 0x 258.00 ns/op 415.00 ns/op 0.62
Object access 1 prop 0.16000 ns/op 0.31600 ns/op 0.51
Map access 1 prop 0.14200 ns/op 0.31000 ns/op 0.46
Object get x1000 6.2400 ns/op 5.1900 ns/op 1.20
Map get x1000 6.5950 ns/op 5.7190 ns/op 1.15
Object set x1000 37.350 ns/op 24.969 ns/op 1.50
Map set x1000 28.560 ns/op 18.655 ns/op 1.53
Return object 10000 times 0.31750 ns/op 0.29570 ns/op 1.07
Throw Error 10000 times 3.5980 us/op 2.7222 us/op 1.32
toHex 159.05 ns/op 112.36 ns/op 1.42
Buffer.from 138.10 ns/op 104.28 ns/op 1.32
shared Buffer 96.112 ns/op 70.093 ns/op 1.37
fastMsgIdFn sha256 / 200 bytes 2.3480 us/op 1.9870 us/op 1.18
fastMsgIdFn h32 xxhash / 200 bytes 249.00 ns/op 392.00 ns/op 0.64
fastMsgIdFn h64 xxhash / 200 bytes 298.00 ns/op 431.00 ns/op 0.69
fastMsgIdFn sha256 / 1000 bytes 7.5900 us/op 5.9840 us/op 1.27
fastMsgIdFn h32 xxhash / 1000 bytes 390.00 ns/op 510.00 ns/op 0.76
fastMsgIdFn h64 xxhash / 1000 bytes 360.00 ns/op 496.00 ns/op 0.73
fastMsgIdFn sha256 / 10000 bytes 67.624 us/op 49.702 us/op 1.36
fastMsgIdFn h32 xxhash / 10000 bytes 1.9290 us/op 1.8870 us/op 1.02
fastMsgIdFn h64 xxhash / 10000 bytes 1.2790 us/op 1.3210 us/op 0.97
send data - 1000 256B messages 11.633 ms/op 9.9779 ms/op 1.17
send data - 1000 512B messages 16.747 ms/op 12.941 ms/op 1.29
send data - 1000 1024B messages 27.569 ms/op 22.195 ms/op 1.24
send data - 1000 1200B messages 20.600 ms/op 23.191 ms/op 0.89
send data - 1000 2048B messages 31.974 ms/op 29.239 ms/op 1.09
send data - 1000 4096B messages 30.867 ms/op 26.728 ms/op 1.15
send data - 1000 16384B messages 71.001 ms/op 63.943 ms/op 1.11
send data - 1000 65536B messages 217.91 ms/op 242.63 ms/op 0.90
enrSubnets - fastDeserialize 64 bits 1.1460 us/op 1.1160 us/op 1.03
enrSubnets - ssz BitVector 64 bits 379.00 ns/op 502.00 ns/op 0.75
enrSubnets - fastDeserialize 4 bits 156.00 ns/op 320.00 ns/op 0.49
enrSubnets - ssz BitVector 4 bits 382.00 ns/op 504.00 ns/op 0.76
prioritizePeers score -10:0 att 32-0.1 sync 2-0 180.81 us/op 128.36 us/op 1.41
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 184.67 us/op 163.57 us/op 1.13
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 359.00 us/op 235.03 us/op 1.53
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 500.50 us/op 419.09 us/op 1.19
prioritizePeers score 0:0 att 64-1 sync 4-1 907.59 us/op 632.54 us/op 1.43
array of 16000 items push then shift 1.7487 us/op 1.2914 us/op 1.35
LinkedList of 16000 items push then shift 7.7950 ns/op 6.3210 ns/op 1.23
array of 16000 items push then pop 132.48 ns/op 78.839 ns/op 1.68
LinkedList of 16000 items push then pop 7.6990 ns/op 6.2430 ns/op 1.23
array of 24000 items push then shift 2.5546 us/op 1.8947 us/op 1.35
LinkedList of 24000 items push then shift 7.7290 ns/op 6.4570 ns/op 1.20
array of 24000 items push then pop 172.69 ns/op 109.00 ns/op 1.58
LinkedList of 24000 items push then pop 8.1730 ns/op 6.4130 ns/op 1.27
intersect bitArray bitLen 8 7.5160 ns/op 5.5110 ns/op 1.36
intersect array and set length 8 53.192 ns/op 40.070 ns/op 1.33
intersect bitArray bitLen 128 35.819 ns/op 26.518 ns/op 1.35
intersect array and set length 128 779.95 ns/op 591.45 ns/op 1.32
bitArray.getTrueBitIndexes() bitLen 128 2.2310 us/op 2.2770 us/op 0.98
bitArray.getTrueBitIndexes() bitLen 248 4.5760 us/op 4.0680 us/op 1.12
bitArray.getTrueBitIndexes() bitLen 512 8.8720 us/op 6.6840 us/op 1.33
Buffer.concat 32 items 1.0440 us/op 998.00 ns/op 1.05
Uint8Array.set 32 items 2.0320 us/op 1.7230 us/op 1.18
Buffer.copy 2.3880 us/op 2.0950 us/op 1.14
Uint8Array.set - with subarray 3.2470 us/op 2.7110 us/op 1.20
Uint8Array.set - without subarray 1.7830 us/op 1.9500 us/op 0.91
getUint32 - dataview 294.00 ns/op 392.00 ns/op 0.75
getUint32 - manual 245.00 ns/op 336.00 ns/op 0.73
Set add up to 64 items then delete first 2.6134 us/op 1.8394 us/op 1.42
OrderedSet add up to 64 items then delete first 4.0378 us/op 2.8008 us/op 1.44
Set add up to 64 items then delete last 3.0455 us/op 2.0570 us/op 1.48
OrderedSet add up to 64 items then delete last 4.1176 us/op 3.0916 us/op 1.33
Set add up to 64 items then delete middle 2.9096 us/op 2.0469 us/op 1.42
OrderedSet add up to 64 items then delete middle 6.6470 us/op 4.4776 us/op 1.48
Set add up to 128 items then delete first 6.6913 us/op 4.0651 us/op 1.65
OrderedSet add up to 128 items then delete first 11.452 us/op 6.3123 us/op 1.81
Set add up to 128 items then delete last 6.5359 us/op 3.9209 us/op 1.67
OrderedSet add up to 128 items then delete last 10.100 us/op 5.9555 us/op 1.70
Set add up to 128 items then delete middle 7.1465 us/op 3.9158 us/op 1.83
OrderedSet add up to 128 items then delete middle 16.912 us/op 11.872 us/op 1.42
Set add up to 256 items then delete first 12.431 us/op 7.9374 us/op 1.57
OrderedSet add up to 256 items then delete first 18.512 us/op 12.529 us/op 1.48
Set add up to 256 items then delete last 12.508 us/op 7.7124 us/op 1.62
OrderedSet add up to 256 items then delete last 16.065 us/op 11.843 us/op 1.36
Set add up to 256 items then delete middle 10.347 us/op 7.6752 us/op 1.35
OrderedSet add up to 256 items then delete middle 46.363 us/op 34.389 us/op 1.35
transfer serialized Status (84 B) 1.6120 us/op 1.4230 us/op 1.13
copy serialized Status (84 B) 1.3960 us/op 1.2080 us/op 1.16
transfer serialized SignedVoluntaryExit (112 B) 1.6110 us/op 1.6160 us/op 1.00
copy serialized SignedVoluntaryExit (112 B) 1.2450 us/op 1.3960 us/op 0.89
transfer serialized ProposerSlashing (416 B) 1.8990 us/op 2.5440 us/op 0.75
copy serialized ProposerSlashing (416 B) 1.7660 us/op 2.5270 us/op 0.70
transfer serialized Attestation (485 B) 1.8110 us/op 2.4570 us/op 0.74
copy serialized Attestation (485 B) 1.6770 us/op 1.9920 us/op 0.84
transfer serialized AttesterSlashing (33232 B) 2.0250 us/op 1.7470 us/op 1.16
copy serialized AttesterSlashing (33232 B) 5.4280 us/op 4.5030 us/op 1.21
transfer serialized Small SignedBeaconBlock (128000 B) 2.8010 us/op 2.4650 us/op 1.14
copy serialized Small SignedBeaconBlock (128000 B) 18.247 us/op 14.637 us/op 1.25
transfer serialized Avg SignedBeaconBlock (200000 B) 3.2330 us/op 2.2800 us/op 1.42
copy serialized Avg SignedBeaconBlock (200000 B) 28.267 us/op 12.804 us/op 2.21
transfer serialized BlobsSidecar (524380 B) 3.8540 us/op 2.2020 us/op 1.75
copy serialized BlobsSidecar (524380 B) 82.416 us/op 72.196 us/op 1.14
transfer serialized Big SignedBeaconBlock (1000000 B) 3.7430 us/op 2.9230 us/op 1.28
copy serialized Big SignedBeaconBlock (1000000 B) 177.74 us/op 141.16 us/op 1.26
pass gossip attestations to forkchoice per slot 3.4397 ms/op 2.4131 ms/op 1.43
forkChoice updateHead vc 100000 bc 64 eq 0 527.82 us/op 413.81 us/op 1.28
forkChoice updateHead vc 600000 bc 64 eq 0 3.8800 ms/op 2.4721 ms/op 1.57
forkChoice updateHead vc 1000000 bc 64 eq 0 6.0140 ms/op 3.9789 ms/op 1.51
forkChoice updateHead vc 600000 bc 320 eq 0 3.3086 ms/op 2.3967 ms/op 1.38
forkChoice updateHead vc 600000 bc 1200 eq 0 3.2480 ms/op 2.4955 ms/op 1.30
forkChoice updateHead vc 600000 bc 7200 eq 0 4.0753 ms/op 2.6407 ms/op 1.54
forkChoice updateHead vc 600000 bc 64 eq 1000 11.735 ms/op 9.3567 ms/op 1.25
forkChoice updateHead vc 600000 bc 64 eq 10000 12.087 ms/op 8.5536 ms/op 1.41
forkChoice updateHead vc 600000 bc 64 eq 300000 14.739 ms/op 11.244 ms/op 1.31
computeDeltas 500000 validators 300 proto nodes 5.1089 ms/op 3.1677 ms/op 1.61
computeDeltas 500000 validators 1200 proto nodes 4.8948 ms/op 3.1904 ms/op 1.53
computeDeltas 500000 validators 7200 proto nodes 4.6492 ms/op 3.2121 ms/op 1.45
computeDeltas 750000 validators 300 proto nodes 6.7307 ms/op 4.8394 ms/op 1.39
computeDeltas 750000 validators 1200 proto nodes 7.0084 ms/op 4.7684 ms/op 1.47
computeDeltas 750000 validators 7200 proto nodes 6.7605 ms/op 4.8081 ms/op 1.41
computeDeltas 1400000 validators 300 proto nodes 12.906 ms/op 8.9304 ms/op 1.45
computeDeltas 1400000 validators 1200 proto nodes 13.390 ms/op 8.8296 ms/op 1.52
computeDeltas 1400000 validators 7200 proto nodes 13.976 ms/op 8.8853 ms/op 1.57
computeDeltas 2100000 validators 300 proto nodes 20.989 ms/op 13.376 ms/op 1.57
computeDeltas 2100000 validators 1200 proto nodes 21.524 ms/op 13.754 ms/op 1.56
computeDeltas 2100000 validators 7200 proto nodes 21.877 ms/op 13.486 ms/op 1.62
altair processAttestation - 250000 vs - 7PWei normalcase 2.1022 ms/op 1.4000 ms/op 1.50
altair processAttestation - 250000 vs - 7PWei worstcase 3.1544 ms/op 2.1208 ms/op 1.49
altair processAttestation - setStatus - 1/6 committees join 129.34 us/op 65.442 us/op 1.98
altair processAttestation - setStatus - 1/3 committees join 205.54 us/op 124.12 us/op 1.66
altair processAttestation - setStatus - 1/2 committees join 296.19 us/op 187.76 us/op 1.58
altair processAttestation - setStatus - 2/3 committees join 352.79 us/op 259.04 us/op 1.36
altair processAttestation - setStatus - 4/5 committees join 512.06 us/op 374.88 us/op 1.37
altair processAttestation - setStatus - 100% committees join 631.67 us/op 447.16 us/op 1.41
altair processBlock - 250000 vs - 7PWei normalcase 4.3178 ms/op 4.5155 ms/op 0.96
altair processBlock - 250000 vs - 7PWei normalcase hashState 28.132 ms/op 24.202 ms/op 1.16
altair processBlock - 250000 vs - 7PWei worstcase 40.088 ms/op 35.969 ms/op 1.11
altair processBlock - 250000 vs - 7PWei worstcase hashState 76.831 ms/op 71.303 ms/op 1.08
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3580 ms/op 1.7882 ms/op 1.32
phase0 processBlock - 250000 vs - 7PWei worstcase 25.777 ms/op 22.245 ms/op 1.16
altair processEth1Data - 250000 vs - 7PWei normalcase 371.59 us/op 255.19 us/op 1.46
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 5.7040 us/op 2.8770 us/op 1.98
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 35.371 us/op 34.094 us/op 1.04
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.190 us/op 8.5090 us/op 1.20
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 6.2900 us/op 7.4350 us/op 0.85
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 159.45 us/op 121.00 us/op 1.32
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.5682 ms/op 782.92 us/op 2.00
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.7287 ms/op 1.1485 ms/op 1.51
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.0479 ms/op 1.1325 ms/op 1.81
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.6334 ms/op 2.9111 ms/op 1.25
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6294 ms/op 1.0950 ms/op 1.49
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.9267 ms/op 2.7518 ms/op 1.43
Tree 40 250000 create 261.62 ms/op 193.12 ms/op 1.35
Tree 40 250000 get(125000) 165.78 ns/op 106.02 ns/op 1.56
Tree 40 250000 set(125000) 721.75 ns/op 512.41 ns/op 1.41
Tree 40 250000 toArray() 21.034 ms/op 12.326 ms/op 1.71
Tree 40 250000 iterate all - toArray() + loop 21.644 ms/op 12.212 ms/op 1.77
Tree 40 250000 iterate all - get(i) 59.811 ms/op 39.589 ms/op 1.51
Array 250000 create 3.7983 ms/op 2.3432 ms/op 1.62
Array 250000 clone - spread 1.4814 ms/op 1.1788 ms/op 1.26
Array 250000 get(125000) 0.45100 ns/op 0.57100 ns/op 0.79
Array 250000 set(125000) 0.46300 ns/op 0.56700 ns/op 0.82
Array 250000 iterate all - loop 104.97 us/op 73.694 us/op 1.42
phase0 afterProcessEpoch - 250000 vs - 7PWei 54.916 ms/op 41.668 ms/op 1.32
Array.fill - length 1000000 3.6370 ms/op 2.4310 ms/op 1.50
Array push - length 1000000 19.303 ms/op 13.833 ms/op 1.40
Array.get 0.28744 ns/op 0.26495 ns/op 1.08
Uint8Array.get 0.45383 ns/op 0.34188 ns/op 1.33
phase0 beforeProcessEpoch - 250000 vs - 7PWei 19.462 ms/op 12.722 ms/op 1.53
altair processEpoch - mainnet_e81889 286.38 ms/op 250.36 ms/op 1.14
mainnet_e81889 - altair beforeProcessEpoch 20.457 ms/op 18.590 ms/op 1.10
mainnet_e81889 - altair processJustificationAndFinalization 16.935 us/op 10.911 us/op 1.55
mainnet_e81889 - altair processInactivityUpdates 7.0695 ms/op 4.2832 ms/op 1.65
mainnet_e81889 - altair processRewardsAndPenalties 40.323 ms/op 49.946 ms/op 0.81
mainnet_e81889 - altair processRegistryUpdates 2.0590 us/op 2.0480 us/op 1.01
mainnet_e81889 - altair processSlashings 386.00 ns/op 777.00 ns/op 0.50
mainnet_e81889 - altair processEth1DataReset 409.00 ns/op 683.00 ns/op 0.60
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.8529 ms/op 965.01 us/op 1.92
mainnet_e81889 - altair processSlashingsReset 4.1730 us/op 1.5170 us/op 2.75
mainnet_e81889 - altair processRandaoMixesReset 5.6260 us/op 3.2270 us/op 1.74
mainnet_e81889 - altair processHistoricalRootsUpdate 487.00 ns/op 707.00 ns/op 0.69
mainnet_e81889 - altair processParticipationFlagUpdates 3.0150 us/op 3.2010 us/op 0.94
mainnet_e81889 - altair processSyncCommitteeUpdates 892.00 ns/op 775.00 ns/op 1.15
mainnet_e81889 - altair afterProcessEpoch 53.880 ms/op 40.709 ms/op 1.32
capella processEpoch - mainnet_e217614 1.1623 s/op 1.0117 s/op 1.15
mainnet_e217614 - capella beforeProcessEpoch 81.067 ms/op 64.863 ms/op 1.25
mainnet_e217614 - capella processJustificationAndFinalization 19.315 us/op 13.019 us/op 1.48
mainnet_e217614 - capella processInactivityUpdates 19.205 ms/op 14.913 ms/op 1.29
mainnet_e217614 - capella processRewardsAndPenalties 257.47 ms/op 258.02 ms/op 1.00
mainnet_e217614 - capella processRegistryUpdates 19.954 us/op 12.298 us/op 1.62
mainnet_e217614 - capella processSlashings 619.00 ns/op 752.00 ns/op 0.82
mainnet_e217614 - capella processEth1DataReset 464.00 ns/op 715.00 ns/op 0.65
mainnet_e217614 - capella processEffectiveBalanceUpdates 16.825 ms/op 5.8999 ms/op 2.85
mainnet_e217614 - capella processSlashingsReset 4.5920 us/op 2.0570 us/op 2.23
mainnet_e217614 - capella processRandaoMixesReset 8.5680 us/op 3.3180 us/op 2.58
mainnet_e217614 - capella processHistoricalRootsUpdate 888.00 ns/op 702.00 ns/op 1.26
mainnet_e217614 - capella processParticipationFlagUpdates 2.1990 us/op 1.7520 us/op 1.26
mainnet_e217614 - capella afterProcessEpoch 132.40 ms/op 101.24 ms/op 1.31
phase0 processEpoch - mainnet_e58758 344.80 ms/op 295.35 ms/op 1.17
mainnet_e58758 - phase0 beforeProcessEpoch 83.705 ms/op 79.572 ms/op 1.05
mainnet_e58758 - phase0 processJustificationAndFinalization 14.203 us/op 11.225 us/op 1.27
mainnet_e58758 - phase0 processRewardsAndPenalties 36.356 ms/op 26.864 ms/op 1.35
mainnet_e58758 - phase0 processRegistryUpdates 9.5450 us/op 3.6130 us/op 2.64
mainnet_e58758 - phase0 processSlashings 400.00 ns/op 722.00 ns/op 0.55
mainnet_e58758 - phase0 processEth1DataReset 426.00 ns/op 727.00 ns/op 0.59
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6648 ms/op 1.9703 ms/op 0.84
mainnet_e58758 - phase0 processSlashingsReset 3.5370 us/op 2.9830 us/op 1.19
mainnet_e58758 - phase0 processRandaoMixesReset 4.1640 us/op 2.9400 us/op 1.42
mainnet_e58758 - phase0 processHistoricalRootsUpdate 341.00 ns/op 701.00 ns/op 0.49
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.5530 us/op 2.6000 us/op 1.37
mainnet_e58758 - phase0 afterProcessEpoch 47.024 ms/op 35.903 ms/op 1.31
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9497 ms/op 1.0369 ms/op 1.88
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.5557 ms/op 1.7230 ms/op 1.48
altair processInactivityUpdates - 250000 normalcase 22.968 ms/op 15.267 ms/op 1.50
altair processInactivityUpdates - 250000 worstcase 20.714 ms/op 17.056 ms/op 1.21
phase0 processRegistryUpdates - 250000 normalcase 8.1960 us/op 3.2820 us/op 2.50
phase0 processRegistryUpdates - 250000 badcase_full_deposits 353.76 us/op 297.02 us/op 1.19
phase0 processRegistryUpdates - 250000 worstcase 0.5 146.31 ms/op 108.53 ms/op 1.35
altair processRewardsAndPenalties - 250000 normalcase 44.877 ms/op 31.691 ms/op 1.42
altair processRewardsAndPenalties - 250000 worstcase 50.054 ms/op 39.978 ms/op 1.25
phase0 getAttestationDeltas - 250000 normalcase 9.8776 ms/op 6.2172 ms/op 1.59
phase0 getAttestationDeltas - 250000 worstcase 8.6212 ms/op 5.8980 ms/op 1.46
phase0 processSlashings - 250000 worstcase 102.39 us/op 81.700 us/op 1.25
altair processSyncCommitteeUpdates - 250000 138.33 ms/op 97.478 ms/op 1.42
BeaconState.hashTreeRoot - No change 257.00 ns/op 430.00 ns/op 0.60
BeaconState.hashTreeRoot - 1 full validator 118.67 us/op 127.77 us/op 0.93
BeaconState.hashTreeRoot - 32 full validator 939.83 us/op 1.1153 ms/op 0.84
BeaconState.hashTreeRoot - 512 full validator 12.098 ms/op 10.137 ms/op 1.19
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 164.76 us/op 155.14 us/op 1.06
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.9396 ms/op 1.9751 ms/op 0.98
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 27.423 ms/op 23.000 ms/op 1.19
BeaconState.hashTreeRoot - 1 balances 94.879 us/op 108.66 us/op 0.87
BeaconState.hashTreeRoot - 32 balances 881.33 us/op 943.98 us/op 0.93
BeaconState.hashTreeRoot - 512 balances 10.471 ms/op 7.3682 ms/op 1.42
BeaconState.hashTreeRoot - 250000 balances 206.21 ms/op 185.91 ms/op 1.11
aggregationBits - 2048 els - zipIndexesInBitList 30.635 us/op 19.124 us/op 1.60
byteArrayEquals 32 67.479 ns/op 47.882 ns/op 1.41
Buffer.compare 32 19.687 ns/op 15.530 ns/op 1.27
byteArrayEquals 1024 1.8216 us/op 1.2700 us/op 1.43
Buffer.compare 1024 30.030 ns/op 24.139 ns/op 1.24
byteArrayEquals 16384 27.398 us/op 20.157 us/op 1.36
Buffer.compare 16384 226.14 ns/op 173.16 ns/op 1.31
byteArrayEquals 123687377 224.28 ms/op 151.83 ms/op 1.48
Buffer.compare 123687377 9.2636 ms/op 3.6907 ms/op 2.51
byteArrayEquals 32 - diff last byte 63.689 ns/op 47.473 ns/op 1.34
Buffer.compare 32 - diff last byte 22.614 ns/op 15.900 ns/op 1.42
byteArrayEquals 1024 - diff last byte 2.0198 us/op 1.2604 us/op 1.60
Buffer.compare 1024 - diff last byte 31.784 ns/op 24.398 ns/op 1.30
byteArrayEquals 16384 - diff last byte 30.282 us/op 20.010 us/op 1.51
Buffer.compare 16384 - diff last byte 225.94 ns/op 202.72 ns/op 1.11
byteArrayEquals 123687377 - diff last byte 253.27 ms/op 151.72 ms/op 1.67
Buffer.compare 123687377 - diff last byte 7.5994 ms/op 3.9692 ms/op 1.91
byteArrayEquals 32 - random bytes 5.8190 ns/op 4.9650 ns/op 1.17
Buffer.compare 32 - random bytes 19.324 ns/op 16.112 ns/op 1.20
byteArrayEquals 1024 - random bytes 5.8890 ns/op 4.8980 ns/op 1.20
Buffer.compare 1024 - random bytes 19.624 ns/op 15.993 ns/op 1.23
byteArrayEquals 16384 - random bytes 6.0100 ns/op 4.9160 ns/op 1.22
Buffer.compare 16384 - random bytes 19.383 ns/op 16.029 ns/op 1.21
byteArrayEquals 123687377 - random bytes 7.5100 ns/op 7.8400 ns/op 0.96
Buffer.compare 123687377 - random bytes 21.430 ns/op 19.510 ns/op 1.10
regular array get 100000 times 38.599 us/op 31.174 us/op 1.24
wrappedArray get 100000 times 38.384 us/op 31.155 us/op 1.23
arrayWithProxy get 100000 times 15.539 ms/op 11.095 ms/op 1.40
ssz.Root.equals 52.930 ns/op 48.185 ns/op 1.10
byteArrayEquals 51.361 ns/op 39.369 ns/op 1.30
Buffer.compare 12.197 ns/op 9.3560 ns/op 1.30
processSlot - 1 slots 18.813 us/op 12.915 us/op 1.46
processSlot - 32 slots 3.5150 ms/op 1.7861 ms/op 1.97
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 46.638 ms/op 37.476 ms/op 1.24
getCommitteeAssignments - req 1 vs - 250000 vc 2.7988 ms/op 1.7676 ms/op 1.58
getCommitteeAssignments - req 100 vs - 250000 vc 5.3626 ms/op 3.5007 ms/op 1.53
getCommitteeAssignments - req 1000 vs - 250000 vc 6.1056 ms/op 3.7226 ms/op 1.64
findModifiedValidators - 10000 modified validators 434.62 ms/op 237.91 ms/op 1.83
findModifiedValidators - 1000 modified validators 221.04 ms/op 157.89 ms/op 1.40
findModifiedValidators - 100 modified validators 177.26 ms/op 140.78 ms/op 1.26
findModifiedValidators - 10 modified validators 199.66 ms/op 141.70 ms/op 1.41
findModifiedValidators - 1 modified validators 188.66 ms/op 142.92 ms/op 1.32
findModifiedValidators - no difference 227.32 ms/op 132.64 ms/op 1.71
compare ViewDUs 3.4403 s/op 3.0401 s/op 1.13
compare each validator Uint8Array 1.7160 s/op 1.7336 s/op 0.99
compare ViewDU to Uint8Array 1.2162 s/op 702.13 ms/op 1.73
migrate state 1000000 validators, 24 modified, 0 new 808.21 ms/op 706.10 ms/op 1.14
migrate state 1000000 validators, 1700 modified, 1000 new 983.34 ms/op 965.47 ms/op 1.02
migrate state 1000000 validators, 3400 modified, 2000 new 1.2723 s/op 1.1850 s/op 1.07
migrate state 1500000 validators, 24 modified, 0 new 753.52 ms/op 746.29 ms/op 1.01
migrate state 1500000 validators, 1700 modified, 1000 new 1.0631 s/op 962.47 ms/op 1.10
migrate state 1500000 validators, 3400 modified, 2000 new 1.4590 s/op 1.1736 s/op 1.24
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 6.0200 ns/op 5.8100 ns/op 1.04
state getBlockRootAtSlot - 250000 vs - 7PWei 996.04 ns/op 1.0510 us/op 0.95
computeProposers - vc 250000 12.035 ms/op 6.6653 ms/op 1.81
computeEpochShuffling - vc 250000 57.918 ms/op 35.751 ms/op 1.62
getNextSyncCommittee - vc 250000 196.40 ms/op 116.53 ms/op 1.69
computeSigningRoot for AttestationData 30.320 us/op 30.032 us/op 1.01
hash AttestationData serialized data then Buffer.toString(base64) 2.0545 us/op 1.2067 us/op 1.70
toHexString serialized data 1.3212 us/op 793.75 ns/op 1.66
Buffer.toString(base64) 273.15 ns/op 145.47 ns/op 1.88
nodejs block root to RootHex using toHex 183.38 ns/op 120.73 ns/op 1.52
nodejs block root to RootHex using toRootHex 115.88 ns/op 80.469 ns/op 1.44
browser block root to RootHex using the deprecated toHexString 371.81 ns/op 205.33 ns/op 1.81
browser block root to RootHex using toHex 285.63 ns/op 167.44 ns/op 1.71
browser block root to RootHex using toRootHex 208.66 ns/op 145.14 ns/op 1.44

by benchmarkbot/action

import {RootHex} from "@lodestar/types";

export enum ArchiveMode {
Full = "full",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you think of a better name for current archiver strategy we use, please suggest.

Copy link

codecov bot commented Oct 17, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 49.19%. Comparing base (245d63e) to head (db03a79).
Report is 17 commits behind head on unstable.

Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #7170      +/-   ##
============================================
- Coverage     49.21%   49.19%   -0.02%     
============================================
  Files           598      598              
  Lines         39726    39779      +53     
  Branches       2093     2097       +4     
============================================
+ Hits          19551    19570      +19     
- Misses        20135    20169      +34     
  Partials         40       40              

packages/cli/src/options/beaconNodeOptions/chain.ts Outdated Show resolved Hide resolved
packages/cli/src/options/beaconNodeOptions/chain.ts Outdated Show resolved Hide resolved
import {RootHex} from "@lodestar/types";

export enum ArchiveMode {
Full = "full",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More of a general question, once we have differential archive mode implemented, will there be any advantage to use our current (full) archiving strategy?

If not, we should probably not make it overly generic / switchable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My idea is to get the differential strategy released as experimental first. As I am a bit skeptical about it's memory implication. So during that time user can choose which strategy to use.

@@ -0,0 +1,46 @@
import {CheckpointWithHex} from "@lodestar/fork-choice";
import {Metrics} from "../../metrics/metrics.js";
import {RootHex} from "@lodestar/types";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the new linter does not enforce import order anymore?

Copy link
Contributor Author

@nazarhussain nazarhussain Oct 21, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a caveat related to import order, not yet enabled. #7182

Copy link
Contributor

@twoeths twoeths left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the approach lgtm for the 1st round, dropped some minor comments

@nazarhussain nazarhussain merged commit c9cf3ea into unstable Oct 21, 2024
20 checks passed
@nazarhussain nazarhussain deleted the nh/diff-feature-flag branch October 21, 2024 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants