-
Notifications
You must be signed in to change notification settings - Fork 3.6k
[fix][broker] Fixes Inconsistent ServiceUnitStateData View (ExtensibleLoadManagerImpl only) #24186
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
… (ExtensibleLoadManagerImpl)
235d094
to
eb0a14d
Compare
...data/src/main/java/org/apache/pulsar/metadata/tableview/impl/MetadataStoreTableViewImpl.java
Outdated
Show resolved
Hide resolved
b25febb
to
7e9e4c5
Compare
7e9e4c5
to
8e43dfb
Compare
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataCache.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR fixes an inconsistent view of ServiceUnitStateData by introducing a no-retry backoff configuration and updating the equality comparison to cover all fields.
- Implements a no-retry backoff configuration in MetadataCacheConfig and applies it in MetadataStoreTableViewImpl.
- Adds tests verifying the new no backoff behavior and enhanced equality checks for ServiceUnitStateData.
- Adjusts Backoff and MetadataCacheImpl behavior to support the no-retry logic.
Reviewed Changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
File | Description |
---|---|
pulsar-metadata/src/test/java/org/apache/pulsar/metadata/MetadataCacheTest.java | Added tests for no backoff configuration in MetadataCacheConfig. |
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/tableview/impl/MetadataStoreTableViewImpl.java | Applied no-retry backoff and adjusted exception handling during put operations. |
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/cache/impl/MetadataCacheImpl.java | Modified exception propagation based on no backoff configuration. |
pulsar-metadata/src/main/java/org/apache/pulsar/metadata/api/MetadataCacheConfig.java | Introduced a static no-retry backoff configuration constant. |
pulsar-common/src/main/java/org/apache/pulsar/common/util/Backoff.java | Updated constructor and reset logic to correctly handle no backoff conditions. |
pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateDataTest.java | Added tests to validate the updated equality behavior of ServiceUnitStateData. |
pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/extensions/channel/ServiceUnitStateData.java | Removed the custom equals override to leverage the default record equality comparing all fields. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #24186 +/- ##
============================================
+ Coverage 73.57% 74.21% +0.64%
+ Complexity 32624 2315 -30309
============================================
Files 1877 1865 -12
Lines 139502 144653 +5151
Branches 15299 16523 +1224
============================================
+ Hits 102638 107354 +4716
+ Misses 28908 28797 -111
- Partials 7956 8502 +546
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
…eLoadManagerImpl only) (apache#24186) (cherry picked from commit c6be44c) (cherry picked from commit 28f7e94)
…eLoadManagerImpl only) (apache#24186) (cherry picked from commit c6be44c) (cherry picked from commit 28f7e94)
Motivation
Although we do CAS updates on ServiceUnitStateMetadataStoreTableViewImpl.put, ServiceUnitStateData.versionId can still conflict if cache.readModifyUpdateOrCreate retries multiple times with the updated value's GetResult.Stats.version.
We should fail-fast if the readModifyUpdateOrCreate fails once(e.g. upon first BadVersionException) not to cause inconsistent ServiceUnitStateData views from different brokers.
Also, to fix the worst case, any inconsistent ServiceUnitStateData views, ServiceUnitStateMetadataStoreTableViewImpl should update the latest value(not only compare the versionId but also all fields) from the MetadataStore by its cache refresh cycles.
Modifications
Verifying this change
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
doc
doc-required
doc-not-needed
doc-complete
Matching PR in forked repository
PR in forked repository: