Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dump fulltext index #2336

Merged
merged 11 commits into from
Dec 7, 2024
Merged

Dump fulltext index #2336

merged 11 commits into from
Dec 7, 2024

Conversation

vsian
Copy link
Contributor

@vsian vsian commented Dec 6, 2024

What problem does this PR solve?

Dump MemoryIndexer when the memory usage of the index exceeds the threshold.

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Test cases

@vsian vsian added the ci PR can be test label Dec 6, 2024
@@ -35,6 +35,11 @@ protected:
auto *memindex_tracer = InfinityContext::instance().storage()->memindex_tracer();
memindex_tracer->AddMemUsed(mem);
}

void DecMemUsed(SizeT mem) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about change the function name to IncreaseMemoryUsage and DecreaseMemoryUsage

@@ -30,7 +30,7 @@ import index_base;
import column_def;
import cleanup_scanner;
import chunk_index_entry;
import memory_indexer;
// import memory_indexer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove the comment

private:
// call with write lock
void AddMemUsed(SizeT mem);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IncreaseMemoryUsage / DecreaseMemoryUsage

@vsian vsian removed the ci PR can be test label Dec 6, 2024
@vsian vsian added the ci PR can be test label Dec 6, 2024
@vsian vsian added ci PR can be test and removed ci PR can be test labels Dec 6, 2024
@JinHai-CN JinHai-CN merged commit 25797ef into infiniflow:main Dec 7, 2024
4 checks passed
@vsian vsian deleted the dump_fulltext_index branch December 10, 2024 04:14
vsian added a commit to vsian/infinity that referenced this pull request Dec 13, 2024
### What problem does this PR solve?

Dump `MemoryIndexer` when the memory usage of the index exceeds the
threshold.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Test cases

---------

Co-authored-by: writinwaters <[email protected]>
@JinHai-CN JinHai-CN mentioned this pull request Jan 10, 2025
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci PR can be test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants