feat: compute receipts root #111

ricomateo · 2024-07-02T18:18:42Z

Closes #106

**Motivation** Replace evm code in eftests with an implementation that uses ethrex types **Description** * Implement `execute_tx` using revm * Move `evm` crate into `core` crate * Move `Transaction` into its own module * Implement multiple getters for `Transaction` fields * Implement address recovery from signed transactions * Implement conversions between ef tests types and ethrex types * Add one more ef test  Closes #21

the value must be rlp(receipt) instead of tx_type || rlp(receipt) (if tx_type != 0)

the previous way in which the receipts trie values were computed was the right way

crates/core/src/types/block.rs

MegaRedHand

LGTM

**Motivation**  **Description**   Closes #issue_number

**Motivation** Obtain world state from account info stored in db **Description** * Implement storage root computation * Implement word state  Closes #44

**Motivation** Add support for EIP2930 transaction **Description** Implements EIP2930Transaction + required methods & traits   Closes None, but will enable #135 to also close #24

This PR is based on #138 (as they modify the same code), please merge it first **Motivation** Add support for EIP4844 transactions  **Description** Add `EIP4844` transactions + needed trait and methods and integrate it into existing transaction-related code   Closes #26 (evm already performs validations before executing so adding the transaction itself will sufice)

**Motivation** Add error handling to `execute_tx` and remove unwrap  **Description** Add `EvmError` Map revm's `EVMError` to `EvmError` Add error handling to `execute_tx`   Closes None

**Motivation**  **Description**   Closes #125 --------- Co-authored-by: Tomás Grüner <[email protected]>

**Motivation** Being able to store and fetch blocks from the db via `Store` api **Description** * Replace `Transaction::encode` with `Transaction::encode_with_type` logic (Reasoning: We cannot decode the transactions without knowing their type with the current behaviour, making it impossible to implement `RLPDecode` for `Transaction`) * Implement `RLPDecode` for `Transaction` (Using the logic that was previoulsy used in `EncodedTransaction::decode` (payload module)) * Implement `RLPDecode` for `Withdrawal`, `BlockHeader` and `BlockBody` * Add the folloewng methods to `Store` and `StoreEngine`: `add_block_header`, `add_block_body`, `get_block_header`, `get_block_body`, and implement them for `InMemory` and `Libmdbx` engine types   Closes None, but is needed for #145

**Motivation** Being able to serialize blocks matching the output required by the rpc spec  **Description** *Add struct `BlockSerializable` which can be serialized to match the specifications of the rpc. It can contain either the full transactions or just their hashes. *Implement serialization for `BlockSerializable`, `BlockHeader`, `BlockBody`, `Withdrawal` and `Transaction`. Other changes: *Replace `Bloom` with `ethereum_types::Bloom` *Rename some fields in `Eip1559Transaction` to match other transaction types' fields   Closes None, but is needed in order to implement #31

**Motivation** - Make running and modifying EF Tests a better experience **Description**  - Remove spinner and just use prints - We now can filter the tests we want to run by fork name(we do the filtering in the parsing). Default is all forks. - Upgrade tests to more recent version - Run some Legacy Tests that we weren't running before. This adds a lot of tests more, it is the folder Cancun under LegacyTests. There will be repeated tests with the folder GeneralStateTests, we may want to find a solution for that so that it takes less time to execute. - Create docs in `README.md` - Implement some nits in the runner, making code easier to understand. - Ignore a few tests that take too long to run so that we can check for breaking changes fast. - Fix comparison report against LEVM, they weren't working correctly mostly because we were mishandling our Cache - Tidy the report, now it is much more clear and easier for debugging. Also the code is easier to follow and more concise too! - Fix some tests with REVM, basically now using constructor of `BlobExcessGasAndPrice` and setting chain id to 1 (as we do in LEVM). - Changed `get_state_transitions` [here](#2518) so that REVM and LEVM account updates are mostly the same and the comparison is more accurate for the person who is debugging any test.  Closes #issue_number

**Motivation**  - merge main changes to my other PR because main had a huge refactor and my PR was also a huge refactor :) **Description**   Closes #issue_number --------- Co-authored-by: Julian Ventura <[email protected]> Co-authored-by: Avila Gastón <[email protected]> Co-authored-by: fmoletta <[email protected]> Co-authored-by: Matías Onorato <[email protected]> Co-authored-by: Edgar <[email protected]> Co-authored-by: Tomás Arjovsky <[email protected]> Co-authored-by: Martin Paulucci <[email protected]> Co-authored-by: Lucas Fiegl <[email protected]> Co-authored-by: Estéfano Bargas <[email protected]> Co-authored-by: Javier Rodríguez Chatruc <[email protected]> Co-authored-by: VolodymyrBg <[email protected]> Co-authored-by: Tomás Paradelo <[email protected]> Co-authored-by: Mauro Toscano <[email protected]> Co-authored-by: Cypher Pepe <[email protected]>

**Motivation** Add instructions on how to set up ethrex along with a consensus node and start syncing with holesky or other testnets  **Description**   Closes #issue_number --------- Co-authored-by: Javier Rodríguez Chatruc <[email protected]>

**Motivation**  Currently the ProverServer component have the responsibility for both act as a server for the ProverClient (i.e., send blocks to prove and receive proofs) and send proofs to the L1 contract to verify blocks. This tasks can be parallel and decoupled one from the other. **Description**  A new struct `L1ProofSender` is created that periodically checks if there're new proofs to send to the L1, removing that job from the `ProverServer`. Also, components were renamed for better clarity. Note that the config names were not changed as there's a WIP PR (#2501) doing a full refactor of it  --------- Co-authored-by: Javier Rodríguez Chatruc <[email protected]> Co-authored-by: Javier Chatruc <[email protected]> Co-authored-by: Ivan Litteri <[email protected]>

**Motivation**  The flamegraphs are displayed half-sized on GH Pages. **Description**  In the CI, remove the line that cause the problem. Added Zed editor config directory to `.gitignore` btw.

**Motivation**  - Temporarily comment Invalid Missing Ancestor Syncing ReOrg and Invlalid P9 and P10. Should be fixed later, the issue is [this one](#2565) **Description**   Closes #issue_number

…0% now (#2568) **Motivation** - Fix CI **Description**  - London tests don't pass 100% because new tests have been added and it seems that there is an edge case we are not currently passing. For now I wanted to disable the check that sees if all tests from Prague to London passed and set it to look only for Prague to Paris. - Added `workflow_dispatch` to this workflow so that we can run it manually.  Closes #issue_number

**Motivation**  - stop popping callframe in finalize execution **Description**   Closes #issue_number

**Motivation** Now that the L2 has fully defaulted to levm, both for execution and proving, these modules became unused. **Description**   Closes #issue_number

Validium is a [scaling solution](https://ethereum.org/en/developers/docs/scaling/) that enforces integrity of transactions using validity proofs like [ZK-rollups](https://ethereum.org/en/developers/docs/scaling/zk-rollups/), but doesn’t store transaction data on the Ethereum Mainnet. **Description** - Replace EIP 4844 transactions for EIP 1559 - Modify OnChainProposer contract so that it supports validium. It is not the most efficient way of doing it but the simplest. - Now the config.toml has a validium field. Note: I'm not 100% sure about the changes that I made to the OnChainProposer, there may be a mistake in the additions that I made. I will review it though but I still consider worth opening this PR.  Closes #2313 --------- Co-authored-by: ilitteri <[email protected]> Co-authored-by: Ivan Litteri <[email protected]>

**Motivation** - We were just checking that the post state root matched. This checks that logs match too. **Description**  - This PR for now just compares the logs root hash with the provided by the `EFTest` but we might also want to compare against REVM's logs so that the log diff is debuggable in a follow up. This has to be added.  Closes #issue_number Co-authored-by: Javier Rodríguez Chatruc <[email protected]>

…sync hive test (#2605) **Motivation** PR #2426 changed how fork choice & new payload interact with the syncer and also introduced a bug. If snap sync is enabled, then fork choice update will never attempt to trigger a sync, so the sync process never gets started. This PR fixes the bug and also refactors the sync manager api to better suit the new use cases  * Combine commonly used together `SyncManager` methods `set_head` & `start_sync` into `sync_to_head` * Remove unused `SyncManager` method `status` and associated struct * Make sure sync is triggered during fcu when needed even if snap sync is enabled * Re-enable snap sync hive test suite **Description**   Closes #2521

…ad endpoint (#2561) **Motivation** This fixes the failing consume-engine hive tests on Prague and Cancun  **Description** * Return `INVALID` payload status when we fail to decode transactions as per spec ([Step 6 item 1](https://github.com/ethereum/execution-apis/blob/main/src/engine/paris.md#engine_newpayloadv1))   Closes #issue_number

**Motivation** There is currently a bug in the storage healer causing fetched paths to not be properly updated. This makes storage healing virtually infinite as fetched paths are constantly being added back to the queue. This fix should restore regular storage healing behaviour  **Description** * Fix logic error when updating pending paths for the next fetch during storage healing  **Other info** This bug was unknowingly introduced by #2288  Closes #issue_number

**Motivation** After recent changes in main, rebuilding now takes a lot longer than state sync. This PR aims to mitigate this hit by introducing other performance upgrades  **Description** * Increase parallelism when rebuilding storages * Reduce intermediate hashing when rebuilding state tries  These changes have increased storage rebuild speed to around the same as before the changes to store, and has reduced time estimates for state rebuild, but doesn't manage to make the state rebuild keep up with the state sync. These changes have not affected state sync speed  Closes #issue_number

…or encoding/decoding (#2454) **Motivation** Implements the refactor specified in the linked issues. Adding a single `code` method for the RLPxMessage enum was not enough for both encoding and decoding (as we would need to create the struct to call the method) so an associated constant was also added to support both needs. This solution fulfills the purpose of the issue, to have only one instance of each message code that we can use for encoding/decoding of messages, but its implementation is more complex than what we would have liked. If the complexity is not acceptable, we should close both this PR and the originating issue.  **Description** * Implement `code` method for `RLPxMessage` * Add `CODE` associated constant to `RLPxEncode` trait   Closes #1035 (Also #1034)

**Motivation**  **Description**   Closes #issue_number --------- Co-authored-by: Javier Chatruc <[email protected]>

**Motivation**  **Description**   Closes #issue_number

…test (#2615) **Motivation** Due to feeding so many txs, the base fee keeps increasing so when it goes beyond the load test txs max fee per gas, the block will have 0 txs due to them all having the same max fee per gas. This pr increasing the load test max fee per has to u64 MAX and lowers priority fee per gas to decreasing (realistically removing) the chance of 0 block txs in load tests **Description**   Closes #2523

**Motivation** - Stop using the Account and AccountInfo types defined in LEVM and start using the ones defined in the L1 client. - Biggest changes are that AccountInfo no longer has code, so we can't use it with that purpose and also we don't have our struct StorageSlot anymore, so we have to keep track of original values somewhere else. **Description** - Now we use the structs of the L1 client but they are different from the ones that we used so I had to make changes: - `get_account_info` is now `get_account` because we also need the code of the account and `AccountInfo` has the `code_hash` only. This makes changes on every structure that implements `LevmDatabase` trait. - Now that we don't have `StorageSlot` that had the `current_value` and `original_value` of a storage slot (`original_value` being the value pre-tx) I had to make some changes to logic and store those original values into an auxiliary `HashMap` on `VM`. - Added new function `get_original_storage()` for getting the original storage value. - Make some tiny changes in SSTORE, mostly organize it better. Storage changes deep description: - Now every time we want to get the `original_value` we will look up in the original values stored in the VM struct. These intends to store the storage values previous to starting the execution of a particular transaction. For efficiency and performance, we only update this new field when actually getting the original value. - Let me clarify: At the beginning of the transaction the `CacheDB` could have a lot of accounts with their storage but the `VM.storage_original_values`will start empty on every transaction. When `SSTORE` opcode is executed and we actually care for the original value of a storage slot we will look at `storage_original_values` and it won’t find it (the first time), so then it will see what the value in the `CacheDB` is, and if it’s not there it will finally check on the actual `Database`. After retrieving the value, it will be added to `storage_original_values` , but ONLY the FIRST time. That means that if the value keeps on changing the `original_value` won’t change because once it’s added it’s not modified.  Closes #issue_number

**Motivation** Snap sync hive test has been flaky lately, after running with debug output on the CI the problem seems to be a timeout when fetching the latest block. A [PR](lambdaclass/hive#22) was added to our hive fork to extend this timeout so the test doesn't fail. The timeout for the whole sync process has been left unchanged  **Description** * Extends the timeout for fetching the latest block on the sync hive test suite * Update Hive ref   Closes #issue_number

…2597) **Motivation** Allow running hive tests with snap sync using the available Makefile targets and passing the optional SYNCMODE=snap argument  **Description** * Add `SYNCOMDE` variable to Makefile with default value "full" * Set `syncmode` ethrex flag on hive Makefile targets according to above variable   Closes #issue_number

**Motivation** The client version was hardcoded in the rpc crate It was used in the client RPC msg, in the admin_info RPC msg and in the helloMsg in P2P **Description** Added vergen crate to include more environment variables at build time in the ethrex main package. It can be tested using the following cast commands ```shell cast client --rpc-url localhost:8545 cast rpc admin_nodeInfo --rpc-url http://localhost:8545 ``` Modified the `P2PContext` struct to include the client_info Also added it in the struct `RLPxConnection` to pass it to the helloMessage struct when doing the handshake Modified the test to use the functions with a dummy client_info The version can now be retrieved by using ethrex --version  Closes #2548

**Motivation**  - We were just propagating Internal errors but we also want to propagate DatabaseErrors. Before this we were just reverting the transaction and that is wrong. **Description**   Closes #issue_number

**Motivation** The `Store` methods `set_latest_valid_ancestor` & `get_latest_valid_ancestor` can be confusing without proper documentation. These methods were properly documented on the `StoreEngine` trait, but they were not documented in the `Store` structure where they will be most often called from. This PR adds documentation for these methods on the `Store` implementation while also simplifying it, as the internal trait documentation provides more information on the context and design choices/requirements for the implementation which are not necessary for the top-level methods.  **Description** * Add doc comments for `Store` methods `set_latest_valid_ancestor` & `get_latest_valid_ancestor`   Closes None

**Motivation**  - Try to remove `account_exists` if possible because it adds complexity and unnecessary checks to the DB. - Try to finally remove `get_account_no_push_cache`, which is related to the previous thing too. **Description** - We now ignore a specific test because [EIP-7702 spec has changed](ethereum/EIPs#9710) and we no longer need to check if the account exists in the trie. - Remove `Option` from `specific_tests` - Remove `get_account_no_push_cache` and the usage of `account_exists` in LEVM. This method is not deleted from the Database because it's used in `get_state_transitions`, and even here it could be removed but I think it is better to keep it in this PR and maybe decide later what to do with this function. (If we remove it it wouldn't make a difference to the state though). - We were able to remove a SpuriousDragon check because we don't support pre-merge forks now Note: `account_exists` hasn't been completely removed from `Database` because we use it in `get_state_transitions` but that is going to change soon and we'll be able to remove it.  Closes #issue_number

**Motivation** Some levm validation errors use the enum variant's name as display message instead of displaying a proper error message.  **Description** * Add error messages for some levm validation errors   Closes #issue_number

**Motivation**  - #2380 - #2609 **Description**   - Adds CLI options for the contract deployer bin and the system contracts updater bin (removing the need of a config file). - Updates the L2 Makefile. - Updates the Docker Compose files. - Updates the `pr-main_l2` workflow. - Updates the L2 integration test. - Removes the `sequencer_config_example.toml` since it is not needed anymore. - Refactors the `crates/l2/contracts` module - Renames the crate from `ethrex-l2_deployer` to `ethrex-l2_contracts`. - Adds a `bin` module with the bins `ethrex_l2_l1_deployer` and `ethrex_l2_system_contracts_updater`. - All the SDK-related logic was moved to the SDK lib. - Cleans up the logic related to the config and toml parsing since now the only bin relying on the config is the Prover. Everything relative to the sequencer was removed, and now it is "hardcoded" for the Prover. Closes #2609. --------- Co-authored-by: Manuel Iñaki Bilbao <[email protected]>

**Motivation** Some hive devp2p tests have been failing as of lately. Particularly the `discv4/Amplification/WrongIP` test. Upon further investigation it looks like the test was previously passing but not for the right reasons. The test consists of sending Ping and Pong messages to the node from a given IP and then sending a `FindNode` request from the same node id but a different IP. The test fails if the node replies with a `Neighbours` message instead of noticing the IP mismatch that could represent an amplification attack. Our test used to pass, but not due to the node catching the potential attack but due to a failure to deliver the neighbors message. On both failing and non-failing attempts the node constructs the neighbors message and attempts to send it which is not the correct behaviour. This PR fixes this problem by checking that the IP from which we received the `FindNode` request matches the ip we stored when validating the node (via ping pong messages) as to prevent amplification attacks. It also adds some doc about the potential attack (taken from geth implementation)  **Description** * Check that the IP from which we receive a FindNode message matches the IP of the node * Add doc about potential amplification attacks on FindNode requests   Closes #issue_number

MegaRedHand and others added 10 commits July 1, 2024 13:03

feat: compute transactions root for a block

a685ff2

fix: was encoding LegacyTransaction incorrectly

611bc86

fix: RLP encode index

49f0374

Merge branch 'main' into compute-transactions-root

7080893

Merge branch 'main' into compute-transactions-root

5f9e6eb

refactor: add Transaction::encode_with_type

a81897f

refactor: add TxType enum for encoding tx type

b966cf4

refactor: add TxKind struct

2c59b6d

Merge branch 'main' into compute-transactions-root

830ecac

add receipts root computation

67f1db0

ricomateo added 4 commits July 2, 2024 17:07

fix: the receipt trie value was being computed wrong

3f2dac6

the value must be rlp(receipt) instead of tx_type || rlp(receipt) (if tx_type != 0)

fix: compute receipts trie value correctly

8e23699

the previous way in which the receipts trie values were computed was the right way

remove unnecessary constructor

3b32d29

merge main

ff376e1

ricomateo marked this pull request as ready for review July 2, 2024 20:59

ricomateo requested a review from a team as a code owner July 2, 2024 20:59

MegaRedHand reviewed Jul 2, 2024

View reviewed changes

crates/core/src/types/block.rs Outdated Show resolved Hide resolved

fix: move compute_receipts_root out of BlockBody impl

f02abf1

MegaRedHand approved these changes Jul 3, 2024

View reviewed changes

MegaRedHand merged commit b3a16bf into main Jul 3, 2024
3 checks passed

MegaRedHand deleted the compute-receipts-root branch July 3, 2024 13:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: compute receipts root #111

feat: compute receipts root #111

ricomateo commented Jul 2, 2024

MegaRedHand left a comment

feat: compute receipts root #111

feat: compute receipts root #111

Conversation

ricomateo commented Jul 2, 2024

MegaRedHand left a comment

Choose a reason for hiding this comment