feat: Refactor event processing code #2152
Conversation
There was a problem hiding this comment.
Thank you so much for the reorganization. This generally LGTM. I have a few more detailed questions.
Note that there is an E2E test and some simtests that are failing currently.
I didn't really look at the moved code; my assumption is that it is essentially unchanged. Let me know if I should review something in a bit more detail.
| @@ -16,6 +16,9 @@ pub use node::StorageNodeConfigLoader; | |||
| #[cfg(feature = "node")] | |||
| pub use node::errors::SyncNodeConfigError; | |||
|
|
|||
| #[cfg(feature = "node")] | |||
| pub mod event; | |||
There was a problem hiding this comment.
Question: (Why) does this have to be pub?
There was a problem hiding this comment.
ok it doesn't i think, will fix
| }; | ||
| let system_config = crate::node::events::event_processor::SystemConfig { | ||
| let processor_config = | ||
| crate::event::event_processor::config::EventProcessorRuntimeConfig { |
There was a problem hiding this comment.
Question (nit): Is there a reason to qualify this instead of importing as we do normally (also the SystemConfig below)?
| impl EventProcessor { | ||
| /// Creates a new event processor. | ||
| pub async fn new( | ||
| config: EventProcessorConfig, |
There was a problem hiding this comment.
Question: Why was this changed from a reference to an owned parameter?
There was a problem hiding this comment.
ah my bad, will fix
There was a problem hiding this comment.
Thanks @sadhansood for the refactor, the tests are failing, otherwise, this looks good to me. I didn't dive into the code, assuming that they are no logic changes, focused on the structure changes.
Let me know if there are code pieces that I should pay more attention to.
Thank tou Markus, I will look at the failing tests and fix them. There is no net new functionality, so it must be related to the refactor. |
Description
We are looking to move event processing from running on individual storage node to its own dedicated centralized service. This service will be used by storage nodes to read events (and authenticate them) from instead of locally doing checkpoint processing. With that goal in mind, this is the first PR which moves event processing code from
nodeand also refactors it for easier usage. NOTE: there is no functionality that is getting added or removed in this PR.Test plan
Existing tests.
Release notes
Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.
For each box you select, include information after the relevant heading that describes the impact of your changes that
a user might notice and any actions they must take to implement updates. (Add release notes after the colon for each item)