Skip to content

Commit d46a1fe

Browse files
JereSaloJulianVenturaavilagaston9fmolettamationorato
authored
refactor(levm): merge main to cache_rollback (#2557)
**Motivation** <!-- Why does this pull request exist? What are its goals? --> - merge main changes to my other PR because main had a huge refactor and my PR was also a huge refactor :) **Description** <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #issue_number --------- Co-authored-by: Julian Ventura <[email protected]> Co-authored-by: Avila Gastón <[email protected]> Co-authored-by: fmoletta <[email protected]> Co-authored-by: Matías Onorato <[email protected]> Co-authored-by: Edgar <[email protected]> Co-authored-by: Tomás Arjovsky <[email protected]> Co-authored-by: Martin Paulucci <[email protected]> Co-authored-by: Lucas Fiegl <[email protected]> Co-authored-by: Estéfano Bargas <[email protected]> Co-authored-by: Javier Rodríguez Chatruc <[email protected]> Co-authored-by: VolodymyrBg <[email protected]> Co-authored-by: Tomás Paradelo <[email protected]> Co-authored-by: Mauro Toscano <[email protected]> Co-authored-by: Cypher Pepe <[email protected]>
1 parent 27ae20b commit d46a1fe

File tree

132 files changed

+30092
-2841
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

132 files changed

+30092
-2841
lines changed

‎.github/scripts/flamegraph_watcher.sh

100644100755
+17-20
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,28 @@
11
#!/bin/bash
2+
set -e
23

3-
# This script sends 171 * <iterations> transactions to a test account, per defined private key
4-
# then polls the account balance until the expected balance has been reached
5-
# and then kills the process. It also measures the elapsed time of the test and
6-
# outputs it to Github Action's outputs.
7-
iterations=3500
8-
value=1
9-
account=0x33c6b73432B3aeA0C1725E415CC40D04908B85fd
10-
end_val=$((171 * $iterations * $value))
4+
# This script runs a load test and then kills the node under test. The load test sends a
5+
# transaction from each rich account to a random one, so we can check their nonce to
6+
# determine that the load test finished.
7+
#
8+
# Usage:
9+
# ./flamegraph_watcher.sh
10+
# Requires a PROGRAM variable to be set (e.g. ethrex). This $PROGRAM will be killed when the
11+
# load test finishes. Must be run from the context of the repo root.
1112

12-
start_time=$(date +%s)
13-
ethrex_l2 test load --path /home/runner/work/ethrex/ethrex/test_data/private_keys.txt -i $iterations -v --value $value --to $account >/dev/null
13+
# TODO(#2486): Move this to a cached build outside.
14+
echo "Building load test"
15+
cargo build --release --manifest-path ./cmd/load_test/Cargo.toml
1416

15-
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
16-
echo "balance: $output"
17-
while [[ $output -lt $end_val ]]; do
18-
sleep 2
19-
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
20-
echo "balance: $output"
21-
done
17+
echo "Starting load test"
18+
start_time=$(date +%s)
19+
RUST_BACKTRACE=1 ./target/release/load_test -k ./test_data/private_keys.txt -t eth-transfers -N 1000 -n http://localhost:1729 -w 5 >/dev/null
2220
end_time=$(date +%s)
23-
elapsed=$((end_time - start_time))
2421

22+
elapsed=$((end_time - start_time))
2523
minutes=$((elapsed / 60))
2624
seconds=$((elapsed % 60))
27-
output=$(ethrex_l2 info -b -a $account --wei 2>&1)
28-
echo "Balance of $output reached in $minutes min $seconds s, killing process"
25+
echo "All load test transactions included in $minutes min $seconds s, killing node process."
2926

3027
echo killing "$PROGRAM"
3128
sudo pkill "$PROGRAM"

‎.github/workflows/daily_loc.yaml

+2-3
Original file line numberDiff line numberDiff line change
@@ -55,10 +55,9 @@ jobs:
5555
${{ github.event_name == 'workflow_dispatch'
5656
&& secrets.TEST_CHANNEL_SLACK
5757
|| format(
58-
'{0} {1} {2}',
58+
'{0} {1}',
5959
secrets.ETHREX_L1_SLACK_WEBHOOK,
60-
secrets.ETHREX_L2_SLACK_WEBHOOK,
61-
secrets.LEVM_SLACK_WEBHOOK
60+
secrets.ETHREX_L2_SLACK_WEBHOOK
6261
)
6362
}}
6463
run: |

‎.github/workflows/daily_reports.yaml

+7-8
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,9 @@ jobs:
5353
${{ github.event_name == 'workflow_dispatch'
5454
&& secrets.TEST_CHANNEL_SLACK
5555
|| format(
56-
'{0} {1} {2}',
56+
'{0} {1}',
5757
secrets.ETHREX_L1_SLACK_WEBHOOK,
58-
secrets.ETHREX_L2_SLACK_WEBHOOK,
59-
secrets.LEVM_SLACK_WEBHOOK
58+
secrets.ETHREX_L2_SLACK_WEBHOOK
6059
)
6160
}}
6261
run: |
@@ -72,8 +71,8 @@ jobs:
7271
&& secrets.TEST_CHANNEL_SLACK
7372
|| format(
7473
'{0} {1}',
75-
secrets.ETHREX_L2_SLACK_WEBHOOK,
76-
secrets.LEVM_SLACK_WEBHOOK
74+
secrets.ETHREX_L1_SLACK_WEBHOOK,
75+
secrets.ETHREX_L2_SLACK_WEBHOOK
7776
)
7877
}}
7978
run: |
@@ -87,7 +86,7 @@ jobs:
8786
SLACK_WEBHOOK: >
8887
${{ github.event_name == 'workflow_dispatch'
8988
&& secrets.TEST_CHANNEL_SLACK
90-
|| secrets.LEVM_SLACK_WEBHOOK
89+
|| secrets.ETHREX_L1_SLACK_WEBHOOK
9190
}}
9291
# Only send diff message if the diff has changed
9392
run: |
@@ -144,7 +143,7 @@ jobs:
144143
SLACK_WEBHOOK: >
145144
${{ github.event_name == 'workflow_dispatch'
146145
&& secrets.TEST_CHANNEL_SLACK
147-
|| secrets.LEVM_SLACK_WEBHOOK
146+
|| secrets.ETHREX_L1_SLACK_WEBHOOK
148147
}}
149148
run: sh .github/scripts/publish_levm_ef_tests.sh "$SLACK_WEBHOOK"
150149

@@ -163,7 +162,7 @@ jobs:
163162
|| format(
164163
'{0} {1}',
165164
secrets.ETHREX_L2_SLACK_WEBHOOK,
166-
secrets.LEVM_SLACK_WEBHOOK
165+
secrets.ETHREX_L1_SLACK_WEBHOOK
167166
)
168167
}}
169168
run: |

‎.github/workflows/main_flamegraph_report.yaml

+12-51
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,8 @@ permissions:
66
id-token: write
77

88
on:
9-
# re-enable this when fixing the workflow
10-
# push:
11-
# branches: ["main"]
9+
push:
10+
branches: ["main"]
1211
workflow_dispatch:
1312

1413
env:
@@ -183,13 +182,6 @@ jobs:
183182
${{ env.HOME }}/.cargo/bin/inferno-*
184183
key: ${{ runner.os }}-extra-binaries
185184

186-
- name: Cache ethrex_l2
187-
id: cache-ethrex-l2
188-
uses: actions/cache@v4
189-
with:
190-
path: ${{ env.HOME }}/.cargo/bin/ethrex_l2
191-
key: ${{ runner.os }}-ethrex-l2-${{ hashFiles('cmd/ethrex_l2/Cargo.lock') }}
192-
193185
- name: Change perf settings
194186
run: |
195187
sudo sysctl kernel.perf_event_paranoid=-1
@@ -236,30 +228,18 @@ jobs:
236228
echo "$HOME/.cargo/bin/inferno-collapse-perf" already found
237229
fi
238230
239-
- name: Install ethrex_l2 cli
240-
run: |
241-
if [ -f "$HOME/.cargo/bin/ethrex_l2" ]; then
242-
echo "$HOME/.cargo/bin/ethrex_l2" already found
243-
else
244-
cargo install --force --path cmd/ethrex_l2
245-
fi
246-
ethrex_l2 config create default --default
247-
ethrex_l2 config set default
248-
249231
# By default ethrex uses revm as evm backend.
250232
- id: generate-flamegraph-ethrex
251233
name: Generate Flamegraph data for Ethrex
252234
shell: bash
253235
run: |
254236
rm -rf target/release/ethrex
237+
cargo build --release --bin ethrex --features dev
255238
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph -c "record -o perf.data -F997 --call-graph dwarf,16384 -g" \
256-
--bin ethrex --features dev -- --dev --network /home/runner/work/ethrex/ethrex/test_data/genesis-l2.json --http.port 1729 >/dev/null &
257-
while [ ! -x "./target/release/ethrex" ]; do
258-
echo "Waiting for ethrex binary to be ready..."
259-
sleep 2
260-
done
239+
--bin ethrex --release --features dev -- \
240+
--dev --network /home/runner/work/ethrex/ethrex/test_data/genesis-l2-ci.json --http.port 1729 >/dev/null &
261241
sleep 10
262-
echo "Compilation finished. Executing load test..."
242+
echo "Executing load test..."
263243
bash /home/runner/work/ethrex/ethrex/.github/scripts/flamegraph_watcher.sh &&
264244
echo "Load test finished"
265245
@@ -293,12 +273,13 @@ jobs:
293273
with:
294274
toolchain: ${{ env.RUST_RETH_VERSION }}
295275

276+
# We need a reth version that requires a rustc version <= 1.82.0
296277
- name: Checkout reth
297278
uses: actions/checkout@v4
298279
with:
299280
repository: paradigmxyz/reth
300281
path: "reth"
301-
ref: main
282+
ref: b2ead06d1d0804101de0d1eb3a070e08d8eab857
302283

303284
- name: Caching
304285
uses: Swatinem/rust-cache@v2
@@ -316,13 +297,6 @@ jobs:
316297
${{ env.HOME }}/.cargo/bin/inferno-*
317298
key: ${{ runner.os }}-extra-binaries
318299

319-
- name: Cache ethrex_l2
320-
id: cache-ethrex-l2
321-
uses: actions/cache@v4
322-
with:
323-
path: ${{ env.HOME }}/.cargo/bin/ethrex_l2
324-
key: ${{ runner.os }}-ethrex-l2-${{ hashFiles('cmd/ethrex_l2/Cargo.lock') }}
325-
326300
- name: Change perf settings
327301
run: |
328302
sudo sysctl kernel.perf_event_paranoid=-1
@@ -369,35 +343,22 @@ jobs:
369343
echo "$HOME/.cargo/bin/inferno-collapse-perf" already found
370344
fi
371345
372-
- name: Install ethrex_l2 cli
373-
run: |
374-
if [ -f "$HOME/.cargo/bin/ethrex_l2" ]; then
375-
echo "$HOME/.cargo/bin/ethrex_l2" already found
376-
else
377-
cargo install --force --path cmd/ethrex_l2
378-
fi
379-
ethrex_l2 config create default --default
380-
ethrex_l2 config set default
381-
382346
- id: generate-flamegraph-reth
383347
name: Build and test reth
384348
shell: bash
385349
# --dev.block-time 1000ms set to 1000ms to match ethrex block generation time
386350
run: |
387351
cd ./reth
388352
rm -rf target/profiling/reth
389-
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph -c "record -o perf.data -F997 --call-graph dwarf,16384 -g" \
390-
--bin reth --profile profiling -- node --chain /home/runner/work/ethrex/ethrex/test_data/genesis-load-test.json --dev \
353+
cargo build --bin reth --profile profiling
354+
CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph -c "record -o perf.data -F997 --call-graph dwarf,16384 -g" --bin reth --profile profiling -- \
355+
node --chain /home/runner/work/ethrex/ethrex/test_data/genesis-l2-ci.json --dev \
391356
--dev.block-time 1000ms --http.port 1729 --txpool.max-pending-txns 100000000 --txpool.max-new-txns 1000000000 \
392357
--txpool.pending-max-count 100000000 --txpool.pending-max-size 10000000000 --txpool.basefee-max-count 100000000000 \
393358
--txpool.basefee-max-size 1000000000000 --txpool.queued-max-count 1000000000 >/dev/null &
394-
while [ ! -x "./target/profiling/reth" ]; do
395-
echo "Waiting for reth binary to be ready..."
396-
sleep 10
397-
done
398359
sleep 30
399360
echo "Executing load test..."
400-
bash /home/runner/work/ethrex/ethrex/.github/scripts/flamegraph_watcher.sh &&
361+
(cd /home/runner/work/ethrex/ethrex; ./.github/scripts/flamegraph_watcher.sh)
401362
echo "Load test finished"
402363
403364
- name: Generate SVG

‎.github/workflows/main_prover.yaml

+9
Original file line numberDiff line numberDiff line change
@@ -71,3 +71,12 @@ jobs:
7171
- name: Ensure admin permissions in _work
7272
if: always()
7373
run: sudo chown admin:admin -R /home/admin/actions-runner/_work/
74+
75+
- name: Report Status
76+
if: always()
77+
uses: ravsamhq/notify-slack-action@v2
78+
with:
79+
status: ${{ job.status }}
80+
notify_when: 'failure'
81+
env:
82+
SLACK_WEBHOOK_URL: ${{ secrets.ETHREX_L2_SLACK_WEBHOOK }}

‎.github/workflows/pr-main_l1.yaml

+7-6
Original file line numberDiff line numberDiff line change
@@ -160,11 +160,11 @@ jobs:
160160
ethrex_flags: ""
161161
- name: "Cancun Engine tests"
162162
simulation: ethereum/engine
163-
test_pattern: "engine-cancun/Blob Transactions On Block 1|Blob Transaction Ordering, Single|Blob Transaction Ordering, Multiple Accounts|Replace Blob Transactions|Parallel Blob Transactions|ForkchoiceUpdatedV3|ForkchoiceUpdatedV2|ForkchoiceUpdated Version|GetPayload|NewPayloadV3 After Cancun|NewPayloadV3 Before Cancun|NewPayloadV3 Versioned Hashes|Incorrect BlobGasUsed|ParentHash equals BlockHash|RPC:|in ForkchoiceState|Unknown SafeBlockHash|Unknown FinalizedBlockHash|Unique|Re-Execute Payload|Multiple New Payloads|NewPayload with|Build Payload with|Re-org to Previously|Safe Re-Org to Side Chain|Transaction Re-Org|Re-Org Back into Canonical Chain, Depth=5|Suggested Fee Recipient Test|PrevRandao Opcode|Fork ID: *|Invalid Missing Ancestor Syncing ReOrg|Request Blob Pooled Transactions"
163+
test_pattern: "engine-cancun/Blob Transactions On Block 1|Blob Transaction Ordering, Single|Blob Transaction Ordering, Multiple Accounts|Replace Blob Transactions|Parallel Blob Transactions|ForkchoiceUpdatedV3|ForkchoiceUpdatedV2|ForkchoiceUpdated Version|GetPayload|NewPayloadV3 After Cancun|NewPayloadV3 Before Cancun|NewPayloadV3 Versioned Hashes|Incorrect BlobGasUsed|ParentHash equals BlockHash|RPC:|in ForkchoiceState|Unknown SafeBlockHash|Unknown FinalizedBlockHash|Unique|Re-Execute Payload|Multiple New Payloads|NewPayload with|Build Payload with|Re-org to Previously|Safe Re-Org to Side Chain|Transaction Re-Org|Re-Org Back into Canonical Chain, Depth=5|Suggested Fee Recipient Test|PrevRandao Opcode|Fork ID: *|Invalid Missing Ancestor Syncing ReOrg|Request Blob Pooled Transactions|Invalid NewPayload, Incomplete Transactions|Invalid P9|Invalid P10|Re-Org Back to Canonical Chain*|Invalid PayloadAttributes*|Invalid NewPayload, VersionedHashes|Invalid NewPayload, Incomplete VersionedHashes|Invalid NewPayload, Extra VersionedHashes|Bad Hash on NewPayload|Unknown HeadBlockHash|In-Order Consecutive Payload Execution|Valid NewPayload->ForkchoiceUpdated|Invalid NewPayload, ParentHash|Syncing=False|Payload Build after New Invalid Payload"
164164
ethrex_flags: ""
165165
- name: "Paris Engine tests"
166166
simulation: ethereum/engine
167-
test_pattern: "engine-api/RPC|Re-org to Previously Validated Sidechain Payload|Re-Org Back into Canonical Chain, Depth=5|Safe Re-Org|Transaction Re-Org|Inconsistent|Suggested Fee|PrevRandao Opcode Transactions|Fork ID|Unknown SafeBlockHash|Unknown FinalizedBlockHash|Unique Payload ID|Re-Execute Payload|Multiple New Payloads|NewPayload with|Payload Build|Build Payload"
167+
test_pattern: "engine-api/RPC|Re-org to Previously Validated Sidechain Payload|Re-Org Back into Canonical Chain, Depth=5|Safe Re-Org|Transaction Re-Org|Inconsistent|Suggested Fee|PrevRandao Opcode Transactions|Fork ID|Unknown SafeBlockHash|Unknown FinalizedBlockHash|Unique Payload ID|Re-Execute Payload|Multiple New Payloads|NewPayload with|Payload Build|Build Payload|Invalid PayloadAttributes|Unknown HeadBlockHash|Valid NewPayload->ForkchoiceUpdated|Invalid NewPayload, ParentHash|Bad Hash on NewPayload|Invalid P9|In-Order Consecutive Payload Execution|Re-Org Back to Canonical Chain"
168168
ethrex_flags: ""
169169
- name: "Engine withdrawal tests"
170170
simulation: ethereum/engine
@@ -174,10 +174,11 @@ jobs:
174174
simulation: ethereum/sync
175175
test_pattern: ""
176176
ethrex_flags: "--syncmode full"
177-
- name: "Sync snap"
178-
simulation: ethereum/sync
179-
test_pattern: ""
180-
ethrex_flags: "--syncmode snap"
177+
# Flaky test, reenable when fixed
178+
# - name: "Sync snap"
179+
# simulation: ethereum/sync
180+
# test_pattern: ""
181+
# ethrex_flags: "--syncmode snap"
181182
steps:
182183
- name: Checkout sources
183184
uses: actions/checkout@v4

‎.github/workflows/pr-main_l2.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ on:
66
branches: ["**"]
77
paths:
88
- "crates/l2/**"
9+
- "test_data/**"
10+
- "crates/blockchain/dev/**"
911
- "crates/vm/levm/**"
1012
- ".github/workflows/pr-main_l2.yaml"
1113

‎.github/workflows/pr-main_l2_contracts.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,8 @@ jobs:
2424
uses: dtolnay/rust-toolchain@stable
2525
- name: Install solc
2626
uses: pontem-network/get-solc@master
27+
with:
28+
version: v0.8.29
2729
- name: Caching
2830
uses: Swatinem/rust-cache@v2
2931
- name: Run test of deployer.rs

‎.github/workflows/pr-main_l2_prover.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,8 @@ on:
66
branches: ["**"]
77
paths:
88
- "crates/l2/prover/**"
9+
- "test_data/**"
10+
- "crates/blockchain/dev/**"
911
- ".github/workflows/pr-main_l2_prover.yaml"
1012

1113
concurrency:

‎.github/workflows/pr-main_l2_prover_nightly.yaml renamed to ‎.github/workflows/pr-main_l2_prover_nightly.yaml.disabled

+4
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
# DISABLED: Temporarily disabled due to Pico dependency issues
2+
13
# The reason this exists is because the Pico zkVM compiles in nightly only.
24
name: L2 Prover (nightly)
35
on:
@@ -7,6 +9,8 @@ on:
79
branches: ["**"]
810
paths:
911
- "crates/l2/prover/**"
12+
- "test_data/**"
13+
- "crates/blockchain/dev/**"
1014
- "crates/vm/levm/**"
1115
- ".github/workflows/pr-main_l2_prover_nightly.yaml"
1216

‎.github/workflows/pr-main_levm.yaml

+2
Original file line numberDiff line numberDiff line change
@@ -322,6 +322,8 @@ jobs:
322322

323323
- name: Install solc
324324
uses: pontem-network/get-solc@master
325+
with:
326+
version: v0.8.29
325327

326328
- name: Run benchmarks
327329
run: |

‎.github/workflows/pr_perf_levm.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,8 @@ jobs:
3535

3636
- name: Install solc
3737
uses: pontem-network/get-solc@master
38+
with:
39+
version: v0.8.29
3840

3941
- name: Run PR benchmarks
4042
run: |
@@ -89,6 +91,8 @@ jobs:
8991

9092
- name: Install solc
9193
uses: pontem-network/get-solc@master
94+
with:
95+
version: v0.8.29
9296

9397
- name: Run main benchmarks
9498
run: |

‎CHANGELOG.md

+8
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,16 @@
22

33
## Perf
44

5+
### 2025-04-22
6+
7+
- Avoid calculating state transitions after every block in bulk mode [2519](https://github.com/lambdaclass/ethrex/pull/2519)
8+
9+
- Transform the inlined variant of NodeHash to a constant sized array [2516](https://github.com/lambdaclass/ethrex/pull/2516)
10+
511
### 2025-04-11
612

13+
- Removed some unnecessary clones and made some functions const: [2438](https://github.com/lambdaclass/ethrex/pull/2438)
14+
715
- Asyncify some DB read APIs, as well as its users [#2430](https://github.com/lambdaclass/ethrex/pull/2430)
816

917
### 2025-04-09

‎Makefile

+4-4
Original file line numberDiff line numberDiff line change
@@ -161,16 +161,16 @@ start-node-with-flamegraph: rm-test-db ## 🚀🔥 Starts an ethrex client used
161161
--datadir test_ethrex
162162

163163
load-test: ## 🚧 Runs a load-test. Run make start-node-with-flamegraph and in a new terminal make load-node
164-
cargo run --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t eth-transfers
164+
cargo run --release --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t eth-transfers
165165

166166
load-test-erc20:
167-
cargo run --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t erc20
167+
cargo run --release --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t erc20
168168

169169
load-test-fibonacci:
170-
cargo run --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t fibonacci
170+
cargo run --release --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t fibonacci
171171

172172
load-test-io:
173-
cargo run --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t io-heavy
173+
cargo run --release --manifest-path ./cmd/load_test/Cargo.toml -- -k ./test_data/private_keys.txt -t io-heavy
174174

175175
rm-test-db: ## 🛑 Removes the DB used by the ethrex client used for testing
176176
sudo cargo run --release --bin ethrex -- removedb --force --datadir test_ethrex

0 commit comments

Comments
 (0)