refactor(levm): merge main to cache_rollback #2557
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
**Motivation** We want to run the Ethereum Foundation blockchain tests with LEVM. Currently, we are only running them with REVM. **Description** This PR modifies the EF tests runner so it executes the EF tests with both VMs. The implementation combines both executions on the same command `cargo test`, but could be easily modified to include a feature flag to separate both executions if that's desired.
**Motivation** A new version of the Solidity compiler ([v0.8.29](https://github.com/ethereum/solidity/releases/tag/v0.8.29)) has been released. This update modifies the bytecode format, which causes changes in the `genesis-l2.json` file when launching L2 with `make init`. **Description** - Pin Solidity version `0.8.29` in our contracts. - Updates `genesis-l2.json`. Closes None
**Motivation**
With our current implementation, if a block state diff exceeds the blob
size, we are unable to commit that block.
**Description**
This PR provides an initial solution by calculating the state diff size
after including each transaction in the block. If the size exceeds the
blob limit, the previous state is restored and transactions continue to
be added from the remaining senders in the mempool.
**Observations**
- `blockchain::build_payload` was "replicated" in
`block_producer/payload_builder.rs` with two key differences:
- It doesn't call `blockchain::apply_system_operations`.
- It uses a custom L2 implementation of `fill_transactions` which adds
the logic described above.
- Some functions in `blockchain` are now public to allow access from
`payload_builder`.
- `PayloadBuildContext` now contains am owned `Block` instead of a
mutable reference of it, as we need to be able to clone the
`PayloadBuildContext` to restore a previous state.
- `PayloadBuildContext` is cloned before each transaction execution,
which may impact performance.
- After each transaction, `vm.get_state_transitions()` is called to
compute the state diff size.
- Since `REVM::vm.get_state_transitions()` mutates the
`PayloadBuildContext`, we need to clone it to avoid unintended changes.
- An `accounts_info_cache` is used to prevent calling `get_account_info`
on every tx execution.
> [!WARNING]
> - **REVM**: Payload building is **10x slower** due to frequent
`clone()` calls.
> - **LEVM**: Payload building is **100x slower** because
`LEVM::get_state_transitions` internally calls `get_account_info`.
>
> *These issues will be addressed in future PRs.*
…2464) **Motivation** Currenlty, when state healing finishes, the storage healer is cancelled, even if there are still storages to be fetched and the tries are not stale yet, which slows down storage healing due to the frequent restarts if state healing ends before storage healing. This PR aims to fix this by changing the behaviour to not cancel storage healing if state healing is complete, but to instead give this information to the storage healer via an AtomicBoo, so that it can decide whether to stop based on if the state has become stale, or if no more paths are left to heal and mo more will be added by the state healer. <!-- Why does this pull request exist? What are its goals? --> **Description** * Add AtomicBool param to `storage_healer` to signal whether state healing has ended * Don't cancel storage healing if state healer ended due to state healing being complete * End storage healer if there are no more pending storages left & state healing has ended * (Misc) Remove outdated comment <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #issue_number
**Motivation** We are not able to perform estimations on when healing will end, but we should also not stay completely silent while healing takes place, as this is not very user-friendly. This PR aims to add messages to inform wether state and storage healing are taking place, at the same pace as we show state sync and rebuild progress. For state sync, pending paths will be shown. These can give an insight on the progress, as the amount of paths will continuously increase as we progress through the trie, but will start gradually decreasing as we near the end of healing. For storages it is a slightly different story as we don't have the full number of pending storages available for showing so we only show the storages currently in the queue. <!-- Why does this pull request exist? What are its goals? --> **Description** * Periodically show amount of paths left during State Healing * Periodically show storages in queue during Storage Healing <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 -->
**Motivation** Adds mainnet bootnodes & genesis file so we can connect to mainnet by passing `--network mainnet` <!-- Why does this pull request exist? What are its goals? --> **Description** * add mainnet bootnodes & genesis file * recognize mainnet as preset network * update docs <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Contributes to #72 **Notes** Mainnet genesis file is quite large, it is copied from [here](https://github.com/eth-clients/mainnet/blob/main/metadata/genesis.json)
…2456) **Motivation** Replaces `assert_eq` usage with proper errors in rlpx frame decoding <!-- Why does this pull request exist? What are its goals? --> **Description** * Add error variant `RLPXError::InvalidMessageFrame` * Remove usage of `assert_eq` in `RLPxCodec::decode` impl <!-- A clear and concise general description of the changes this PR introduces --> <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #1748
**Description** This PR removes logs from the `ExecutionReport` when a transaction reverts. With this fix, LEVM no longer breaks at block 80k while syncing on Holesky testnet.
**Motivation** devops **Description** add the following enviroment variables to customize the l2 init command. Specially useful for systemd or docker services ``` ETHREX_NETWORK ETHREX_DATADIR ETHREX_METRICS_PORT ETHREX_EVM ETHREX_HTTP_ADDR ETHREX_HTTP_PORT ```
…nst (#2438) **Motivation** Increase perfomance, improve code. <!-- Why does this pull request exist? What are its goals? --> **Description** Some methods took Vec by value just to take it's length, requiring a costly clone each time. Some methods could be made const, if the compiler can make use of this it may increase perfomance. Changed a drain to a into_iter, which is simpler and has better perfomance. Aided by the following clippy command: ``` cargo clippy --all-features -- -D clippy::perfomance -D clippy::nursery -A clippy::use_self -A clippy::too_long_first_doc_paragraph -A clippy::derive_partial_eq_without_eq -A clippy::option_if_let_else ```
Changes: - Flamegraph Watcher srcript now: - Uses the new load test. - Fails if any line fails (e.g. the load test binary panics). - CI: - The flamegraphs are now updated on push to main again. - Compilation and running is separated to delete the "while not compiled" polling. - Reth version is pinned so it does not rely on 2024 features and can be compiled again. - Load test: - `make` targets now run in release mode. - now waits until all transactions are included before exciting. There's a flag to set a timeout. - All ethrex_l2 references are deleted from CI and the watcher. Closes #2466
This PR adds logs for each imported block: - Transaction count - ms/Ggas for execution - ms/Ggas for storage - Percentage between execution and storage in an imported block
**Description** - standarize revm/levm behaviour when importing blocks - Remove fork choice when importing blocks. - Move block importing out of blockchain module
**Motivation** The L1 slack channel is now used for LEVM development **Description** This PR replaces the LEVM webhook with the L1 webhook.
**Motivation** Dev mode is currently broken. **Description** - Changed the block producer so use the `v4` methods that are used in Prague. - Improved error messages Closes #2376
**Motivation** Allows seeing the status of the test network. **Description** - Implements a net_peerCount dummy (required to enable several ethereum-metrics-exporter modules) - Enables ethereum-metrics-exporter with a grafana dashboard in for the kurtosis localnet Note: prometheus v3 must be used since the prometheus kurtosis package adds fallback_scrape_protocol. Closes #2317
**Motivation** Prover's execution was failing because of wrong data in the ExecutionDB. There were also some cases where data was missing but the current tests didn't catch it. **Description** - fixes saving final storage values instead of initial ones. - fixes saving only touched storage values, instead of read ones too. - removes unused `new_store` - simplifies code
**Motivation** Fix prover tests in CI **Description** #2481 broke the L2 tests since they were using a Cancun genesis, and the block producer has switched to Prague
…2426) **Motivation** We should be able to do payload validations even when the node is in a sync process (except if it's snap sync). **Description** - Refactored some code to make it flatter - Removed early return when the node is syncing - minor renames for clarity sake.
**Motivation** The `proposer.md` file was renamed to `sequencer.md`, and the old link in the docs index became obsolete. **Description** Hi! I removed the outdated reference to `proposer.md` in `crates/l2/docs/README.md` since it's now covered under `sequencer.md`. ref: https://github.com/lambdaclass/ethrex/pull/2269/files#diff-95ad85cd4c72b932973f93785a8a1f365b56757d2972fe671ff33221b7bd0546 Co-authored-by: Javier Rodríguez Chatruc <[email protected]>
**Motivation** In #2397, we are having issues with Pico dependencies. Since we are not using it at the moment, we prefer to temporarily disable the job until we focus on it later. **Description** - Disable pico job renaming `pr-main_l2_prover_nightly.yaml` to `.github/workflows/pr-main_l2_prover_nightly.yaml.disabled`. - Create #2550. Closes None
Lines of code reportTotal lines added: Detailed view |
Benchmark for 6703010Click to view benchmark
|
) **Motivation** Currently during batch processing, the state transitions are calculated for every block and then merged, when it would be more performant to calculate them once at the end. **Description** This PR removes the account updates from the execution result and makes every consumer manually request them. <!-- Link to issues: Resolves #111, Resolves #222 --> Closes #2504
This reverts commit d8fea1c.
Benchmark for ff5dd87Click to view benchmark
|
EF Tests Comparison
|
Benchmark for 2a8c1a9Click to view benchmark
|
Benchmark Results ComparisonPR ResultsBenchmark Results: Factorial
Benchmark Results: Factorial - Recursive
Benchmark Results: Fibonacci
Benchmark Results: ManyHashes
Benchmark Results: BubbleSort
Benchmark Results: ERC20 - Transfer
Benchmark Results: ERC20 - Mint
Benchmark Results: ERC20 - Approval
Main ResultsBenchmark Results: Factorial
Benchmark Results: Factorial - Recursive
Benchmark Results: Fibonacci
Benchmark Results: ManyHashes
Benchmark Results: BubbleSort
Benchmark Results: ERC20 - Transfer
Benchmark Results: ERC20 - Mint
Benchmark Results: ERC20 - Approval
|
Benchmark for 2a5682aClick to view benchmark
|
Benchmark for ff0daa4Click to view benchmark
|
Benchmark for 43fd747Click to view benchmark
|
Benchmark for f003dcdClick to view benchmark
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Description
Closes #issue_number