[ISSUE #2090]💫Implement PopReviveService#mergeAndRevive⚗️ #2093
Conversation
WalkthroughThe pull request modifies the Changes
Assessment against linked issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
|
🔊@mxsm 🚀Thanks for your contribution🎉！ 💡CodeRabbit(AI) will review your code first🔥！ Note 🚨The code review suggestions from CodeRabbit are to be used as a reference only, and the PR submitter can decide whether to make changes based on their own judgment. Ultimately, the project management personnel will conduct the final code review💥. |
rocketmq-rust-bot
added
AI review first
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2093 +/- ##
==========================================
- Coverage 28.43% 28.39% -0.05%
==========================================
Files 492 492
Lines 69976 70081 +105
==========================================
Hits 19898 19898
- Misses 50078 50183 +105 ☔ View full report in Codecov by Sentry. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (5)
rocketmq-broker/src/processor/processor_service/pop_revive_service.rs (5)
347-350: Consider enhanced error handling.
Invokingmerge_and_reviveis fine, but when errors occur, only logging and continuing may mask persistent failures. Consider retry logic, exponential backoff, or additional monitoring if this error is critical.
635-650: Topic existence check.
Skipping the checkpoint if the topic config is missing makes sense. However, consider whether the topic might appear shortly afterward or should be created automatically.
651-655: Subscription group config check.
Returning immediately on missing subscription group config is valid. If dynamic subscription group creation is intended, capture or queue the checkpoint for retry.
715-718: Unimplemented re_put_ck.
This method is required to handle retries or re-adding checkpoints. Provide at least a basic implementation or panic with a clear message if not yet ready.
720-722: Unimplemented revive_msg_from_ck.
Similar tore_put_ck, this is a placeholder. Implement or remove to prevent unexpected runtime panics if called.
📜 Review details
Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
rocketmq-broker/src/processor/processor_service/pop_revive_service.rs(6 hunks)
🔇 Additional comments (12)
rocketmq-broker/src/processor/processor_service/pop_revive_service.rs (12)
56-56: Dependency import looks good.
No issues found with importingSubscriptionGroupManager, as it is used elsewhere in the code for retrieving subscription group configs.
73-73: Field addition aligns with usage.
Thesubscription_group_managerfield, wrapped in anArc, is appropriate for concurrent environments and is utilized inmerge_and_revive.
86-86: Constructor parameter is coherent.
Passingsubscription_group_manageras anArc<SubscriptionGroupManager<MS>>to the constructor is consistent with the struct’s usage.
104-104: Proper struct initialization.
This line correctly assignssubscription_group_managerduring struct construction, ensuring all fields are fully populated.
596-599: Async signature looks consistent.
Transformingmerge_and_reviveinto an async function allows non-blocking operations. Be sure it is only invoked where async context is available.
600-603: Initialization logic is clear.
Storing old and end offsets and sorting the list keep the method organized and maintain relevant state for subsequent operations.
604-607: Logging for debug clarity.
Logging the size of the checkpoint list provides a quick snapshot for debugging. This is helpful for diagnosing issues.
608-619: First and last checkpoint logging.
Unwrappingsort_list.last().unwrap()is safe under the!sort_list.is_empty()check. Logging these values is useful for operational insight.
620-627: Condition onshould_run_pop_revive.
Breaking out early ifshould_run_pop_reviveflips to false is logical. Ensure the rest of the code handles partial iteration gracefully.
629-633: Time-based break condition might skip needed revivals.
You break ifend_time - pop_check_point.get_revive_time()is too short. This logic may miss some checkpoints if the clock is slightly behind or if certain checkpoints need immediate attention. Confirm this arrangement is intentional.
686-694: Cleanup and offset update.
Removing stale inflight checkpoints and updatingnew_offsetensures correct synchronization with the process. This looks fine overall.
695-713: Offset commit approach is sensible.
Committing only whennew_offset > old_offsetavoids redundant updates. The final assignment toself.revive_offsetandconsume_revive_obj.new_offsetproperly synchronizes the next iteration.
|
|
||
| // may be need to optimize | ||
| let mut remove = vec![]; | ||
| let length = self.inflight_revive_request_map.lock().len(); | ||
| while length - remove.len() > 3 { | ||
| tokio::time::sleep(tokio::time::Duration::from_millis(100)).await; | ||
| let mut inflight_map = self.inflight_revive_request_map.lock(); | ||
| let entry = inflight_map.first_entry().unwrap(); | ||
| let pair = entry.get(); | ||
| let old_ck = entry.key(); | ||
| if !pair.1 && (get_current_millis() - pair.0 as u64 > 30 * 1000) { | ||
| self.re_put_ck(old_ck, pair); | ||
| remove.push(old_ck.clone()); | ||
| info!( | ||
| "stay too long, remove from reviveRequestMap, {}, {:?}, {}, {}", | ||
| pop_check_point.topic, | ||
| pop_check_point.broker_name, | ||
| pop_check_point.queue_id, | ||
| pop_check_point.start_offset | ||
| ); | ||
| } | ||
| } | ||
| let mut inflight_revive_request_map = self.inflight_revive_request_map.lock(); |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Concurrent inflight map handling.
Heavy locking around inflight_revive_request_map may block other tasks. The loop containing a sleep could degrade throughput if many entries accumulate. Consider more efficient concurrency patterns (e.g., a queue of pending entries or asynchronous channels) to avoid repeated sleeps under the lock.
rocketmq-rust-bot
added
approved
Which Issue(s) This PR Fixes(Closes)
Fixes #2090
Brief Description
How Did You Test This Change?
Summary by CodeRabbit
PopReviveServicewith improved subscription group management