Skip to content

refactor(indexes): Optimize RocksDBAddressIndex to handle pagination in O(log n) #978

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 20, 2024

Conversation

pedroferreira1
Copy link
Member

@pedroferreira1 pedroferreira1 commented Mar 15, 2024

Motivation

We had some users with wallets that had many transactions in single addresses, then the API request address_history got really slow.

More details here

Acceptance Criteria

  • Add parameter in the get_sorted_from_key method of tx group index to find the first tx to return the iterator from.
  • Refactor address history API to remove O(n) operation and use rocksdb tree to find tx with hash and improve performance.

Checklist

  • If you are requesting a merge into master, confirm this code is production-ready and can be included in future releases as soon as it gets merged

Copy link

codecov bot commented Mar 15, 2024

Codecov Report

Attention: Patch coverage is 80.00000% with 6 lines in your changes are missing coverage. Please review.

Project coverage is 85.16%. Comparing base (2b32edc) to head (cc331cf).

Files Patch % Lines
hathor/indexes/memory_tx_group_index.py 44.44% 4 Missing and 1 partial ⚠️
...or/wallet/resources/thin_wallet/address_history.py 88.88% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #978      +/-   ##
==========================================
+ Coverage   85.09%   85.16%   +0.06%     
==========================================
  Files         294      294              
  Lines       22789    22797       +8     
  Branches     3432     3434       +2     
==========================================
+ Hits        19393    19415      +22     
+ Misses       2713     2702      -11     
+ Partials      683      680       -3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@pedroferreira1 pedroferreira1 requested a review from glevco March 15, 2024 13:05
glevco
glevco previously approved these changes Mar 15, 2024
@msbrogli msbrogli force-pushed the fix/memory-tx-group-index-wrong-order branch from d3511d4 to a2ee438 Compare March 15, 2024 20:41
Base automatically changed from fix/memory-tx-group-index-wrong-order to master March 15, 2024 20:42
@msbrogli msbrogli dismissed glevco’s stale review March 15, 2024 20:42

The base branch was changed.

msbrogli
msbrogli previously approved these changes Mar 15, 2024
@pedroferreira1 pedroferreira1 dismissed msbrogli’s stale review March 15, 2024 22:12

The merge-base changed after approval.

@pedroferreira1 pedroferreira1 force-pushed the refactor/address-history-api-performance branch from 88ca592 to 103ca7d Compare March 15, 2024 22:12
glevco
glevco previously approved these changes Mar 15, 2024
msbrogli
msbrogli previously approved these changes Mar 19, 2024
@pedroferreira1 pedroferreira1 dismissed stale reviews from msbrogli and glevco via da98aaa March 19, 2024 23:40
@pedroferreira1 pedroferreira1 force-pushed the refactor/address-history-api-performance branch from da98aaa to cc331cf Compare March 19, 2024 23:40
@pedroferreira1 pedroferreira1 changed the title refactor: address history api now uses rocksdb tree to get address in a specific hash without O(n) performance refactor(indexes): Optimize RocksDBAddressIndex to handle pagination in O(log n) Mar 19, 2024
@pedroferreira1 pedroferreira1 merged commit 8efb6d4 into master Mar 20, 2024
@pedroferreira1 pedroferreira1 deleted the refactor/address-history-api-performance branch March 20, 2024 03:39
@jansegre jansegre mentioned this pull request Apr 5, 2024
2 tasks
@jansegre jansegre mentioned this pull request May 8, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants