Skip to content

Largish changes to in-RAM temp buckets #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 14 commits into
base: next
Choose a base branch
from

Conversation

toad
Copy link
Contributor

@toad toad commented Dec 30, 2015

Major changes here:

  • Track last access to any temporary bucket/RAF.
  • Always migrate buckets to disk [configurable period] after last access.
  • Use less memory for temp buckets by default.

(On next we migrate buckets to disk in order of creation when we run out of disk space)

For consideration. Maybe we want to keep only some of this. This needs testing - porting it forward to next was fairly involved. It should improve performance on average new nodes; I believe I tested it quite a bit at the time, but there have been significant changes since then. There may be lock contention issues, or even full blown deadlocks etc; this is for discussion, not for immediate inclusion.

@nextgens
Copy link
Contributor

I'm not sure I get why it's better than the current behaviour. Why would we want to always migrate stuff to disk?

IMHO if there's no memory pressure it's just wasted disk IO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants