[improve][broker] Optimize subscription seek (cursor reset) by timestamp #22792

dao-jun · 2024-05-28T15:17:13Z

Motivation

Pulsar uses binary search to find the message by timestamp, it will reduce the number of iterations to find the message, and make it more efficient and faster.

Even though the current implementation is correct, and using binary search to speed-up, but it's still not efficient enough.
The current implementation is to scan all the ledgers to find the message by timestamp.
This is a performance bottleneck, especially for large topics with many messages.
Say, if there is a topic which has 1m entries, through the binary search, it will take 20 iterations to find the message.

In some extreme cases, it may lead to a timeout, and the client will not be able to seeking by timestamp.

The motivation of this PR is to optimize the finding message by timestamp, to make it more efficient and faster.

Modifications

Before search entires, calculate the start, end position by LedgerInfo#timestamp and only search entries in the range to avoid search the entire ledgers.

Verifying this change

Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

Documentation

doc
doc-required
doc-not-needed
doc-complete

Matching PR in forked repository

PR in forked repository:

codecov-commenter · 2024-05-28T18:25:27Z

Codecov Report

Attention: Patch coverage is 77.77778% with 8 lines in your changes missing coverage. Please review.

Project coverage is 75.66%. Comparing base (bbc6224) to head (512fd0d).
Report is 835 commits behind head on master.

Files with missing lines	Patch %	Lines
...che/bookkeeper/mledger/impl/ManagedCursorImpl.java	75.00%	3 Missing and 2 partials ⚠️
...er/service/persistent/PersistentMessageFinder.java	81.25%	1 Missing and 2 partials ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #22792      +/-   ##
============================================
+ Coverage     73.57%   75.66%   +2.09%     
+ Complexity    32624     5895   -26729     
============================================
  Files          1877     1961      +84     
  Lines        139502   160677   +21175     
  Branches      15299    19557    +4258     
============================================
+ Hits         102638   121583   +18945     
- Misses        28908    30314    +1406     
- Partials       7956     8780     +824

Flag	Coverage Δ
inttests	`29.57% <0.00%> (+4.99%)`	⬆️
systests	`25.62% <0.00%> (+1.30%)`	⬆️
unittests	`75.37% <77.77%> (+2.53%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...a/org/apache/bookkeeper/mledger/ManagedCursor.java	`31.57% <ø> (-3.72%)`	⬇️
...er/service/persistent/PersistentMessageFinder.java	`70.49% <81.25%> (+4.58%)`	⬆️
...che/bookkeeper/mledger/impl/ManagedCursorImpl.java	`82.03% <75.00%> (+2.74%)`	⬆️

... and 703 files with indirect coverage changes

managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java

KannarFr · 2024-07-17T13:52:51Z

Can I help merge this PR?

dao-jun · 2024-07-18T02:11:54Z

Can I help merge this PR?

This PR needs more review

lhotari · 2024-10-14T11:48:26Z

@dao-jun please rebase

# Conflicts: # managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java # managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/ManagedCursorTest.java

dao-jun · 2024-10-17T13:57:34Z

@lhotari Rebased, please help review when you have time

lhotari · 2024-10-17T14:14:29Z

Closed and reopened to get the change to unblock CI.

FlorentinDUBOIS · 2024-11-05T10:24:34Z

Hello folks, do you have any news about this pull request?
We hit this issue on production and we (Clever Cloud) could help to test if necessary.

dao-jun · 2024-11-07T06:04:02Z

close reopen to trigger the CI checks

…amp (#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

KannarFr · 2025-01-09T13:11:55Z

Thanks a lot for the work done here @dao-jun @lhotari!

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]>

lhotari · 2025-01-29T23:57:21Z

@dao-jun There's a bug report that might be related to this change. Do you have a chance to check #23910? Thanks

dao-jun · 2025-01-30T09:06:39Z

@dao-jun There's a bug report that might be related to this change. Do you have a chance to check #23910? Thanks

Will do it after a few days~

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]>

… LTS - Remove `managedLedgerOffloadReadThreads` from apache#24025 - Remove `managedLedgerMaxReadsInFlightPermitsAcquireTimeoutMillis` and `managedLedgerMaxReadsInFlightPermitsAcquireQueueSize` from apache#23901 - Remove `managedLedgerCursorResetLedgerCloseTimestampMaxClockSkewMillis` from apache#22792 The configs above only increase the complexity and are hard to configure. Add more comments to `managedLedgerPersistIndividualAckAsLongArray` from apache#23759. This config was added to keep the compatibility from 3.x or earlier so it has value to retain. 3.x users must configure it with false when upgrading to 4.0.

…amp (#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

poorbarcode · 2025-05-07T02:28:51Z

https://lists.apache.org/thread/jp1rpjqz9cczsro3b36t15z239zqfqmy Cherry-picked

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab) (cherry picked from commit dd4e289)

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab) (cherry picked from commit dd4e289)

Optimize seeking by timestamp

96031b6

dao-jun added area/broker ready-to-test category/performance Performance issues fix or improvements labels May 28, 2024

dao-jun added this to the 3.4.0 milestone May 28, 2024

dao-jun requested a review from codelipenghui May 28, 2024 15:17

dao-jun self-assigned this May 28, 2024

github-actions bot added the doc-not-needed Your PR changes do not impact docs label May 28, 2024

dao-jun mentioned this pull request May 28, 2024

[improve][broker] Optimize seeking by timestamp #22152

Closed

15 tasks

Technoboy- reviewed May 30, 2024

View reviewed changes

managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java Outdated Show resolved Hide resolved

dao-jun added 2 commits May 30, 2024 11:15

Cleanup code

7f6ebb3

Cleanup code

62c9f26

lhotari modified the milestones: 4.0.0, 4.1.0 Oct 11, 2024

lhotari added the release/4.0.1 label Oct 11, 2024

lhotari added the triage/lhotari/important lhotari's triaging label for important issues or PRs label Oct 14, 2024

dao-jun added 2 commits October 17, 2024 20:30

Merge branch 'refs/heads/master' into dev/optimize_seeking

f4eb1d2

# Conflicts: # managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedCursorImpl.java # managed-ledger/src/test/java/org/apache/bookkeeper/mledger/impl/ManagedCursorTest.java

rebase master

512fd0d

lhotari closed this Oct 17, 2024

lhotari reopened this Oct 17, 2024

dao-jun closed this Nov 7, 2024

dao-jun reopened this Nov 7, 2024

lhotari changed the title ~~[improve][broker] Optimize seeking by timestamp~~ [improve][broker] Optimize subscription seek (cursor reset) by timestamp Jan 9, 2025

lhotari merged commit 2eb4eab into apache:master Jan 9, 2025
54 of 55 checks passed

lhotari added release/4.0.2 and removed release/4.0.3 labels Jan 9, 2025

lhotari added a commit that referenced this pull request Jan 9, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

948d251

…amp (#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

lhotari added the cherry-picked/branch-4.0 label Jan 9, 2025

dao-jun deleted the dev/optimize_seeking branch January 9, 2025 13:21

poorbarcode pushed a commit to poorbarcode/pulsar that referenced this pull request Jan 23, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

8cecdc4

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]>

lhotari mentioned this pull request Jan 30, 2025

[Bug] Seek on a timestamp causes the subscription to be resetted to the earliest position. #23910

Closed

3 tasks

hanmz pushed a commit to hanmz/pulsar that referenced this pull request Feb 12, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

a0fca3c

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]>

BewareMyPower mentioned this pull request Feb 26, 2025

[improve][broker] Clean up config changes without a PIP since the 4.0 LTS #24027

Open

4 tasks

poorbarcode mentioned this pull request Apr 26, 2025

[improve][broker]Improve the feature "Optimize subscription seek (cursor reset) by timestamp": search less entries #24219

Merged

4 tasks

poorbarcode pushed a commit that referenced this pull request May 7, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

dd4e289

…amp (#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

poorbarcode pushed a commit that referenced this pull request May 7, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

00ee67d

…amp (#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

poorbarcode added cherry-picked/branch-3.0 cherry-picked/branch-3.3 release/3.0.12 release/3.3.7 labels May 7, 2025

dao-jun added a commit to ascentstream/pulsar that referenced this pull request May 8, 2025

[improve][broker] Optimize subscription seek (cursor reset) by timest…

49291b4

…amp (apache#22792) Co-authored-by: Lari Hotari <[email protected]> (cherry picked from commit 2eb4eab)

dao-jun mentioned this pull request May 8, 2025

[improve][broker] Optimize message TTL check #24271

Merged

15 tasks

lhotari added release/3.3.8 and removed release/3.3.7 labels May 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[improve][broker] Optimize subscription seek (cursor reset) by timestamp #22792

[improve][broker] Optimize subscription seek (cursor reset) by timestamp #22792

Uh oh!

dao-jun commented May 28, 2024 •

edited

Loading

Uh oh!

codecov-commenter commented May 28, 2024 •

edited

Loading

Uh oh!

Uh oh!

KannarFr commented Jul 17, 2024

Uh oh!

dao-jun commented Jul 18, 2024

Uh oh!

lhotari commented Oct 14, 2024

Uh oh!

dao-jun commented Oct 17, 2024

Uh oh!

lhotari commented Oct 17, 2024

Uh oh!

FlorentinDUBOIS commented Nov 5, 2024

Uh oh!

dao-jun commented Nov 7, 2024

Uh oh!

Uh oh!

KannarFr commented Jan 9, 2025

Uh oh!

lhotari commented Jan 29, 2025

Uh oh!

dao-jun commented Jan 30, 2025

Uh oh!

poorbarcode commented May 7, 2025

Uh oh!

Uh oh!

[improve][broker] Optimize subscription seek (cursor reset) by timestamp #22792

[improve][broker] Optimize subscription seek (cursor reset) by timestamp #22792

Uh oh!

Conversation

dao-jun commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Verifying this change

Does this pull request potentially affect one of the following parts:

Documentation

Matching PR in forked repository

Uh oh!

codecov-commenter commented May 28, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

KannarFr commented Jul 17, 2024

Uh oh!

dao-jun commented Jul 18, 2024

Uh oh!

lhotari commented Oct 14, 2024

Uh oh!

dao-jun commented Oct 17, 2024

Uh oh!

lhotari commented Oct 17, 2024

Uh oh!

FlorentinDUBOIS commented Nov 5, 2024

Uh oh!

dao-jun commented Nov 7, 2024

Uh oh!

Uh oh!

KannarFr commented Jan 9, 2025

Uh oh!

lhotari commented Jan 29, 2025

Uh oh!

dao-jun commented Jan 30, 2025

Uh oh!

poorbarcode commented May 7, 2025

Uh oh!

Uh oh!

dao-jun commented May 28, 2024 •

edited

Loading

codecov-commenter commented May 28, 2024 •

edited

Loading