Skip to content

Revisit the stashing logic in MergeBatcherColumnation #426

Open
@antiguru

Description

@antiguru

As surfaced recently, the stashing logic to recycle empty buffers in the merge batcher is subtle; it should make sure there are at most 2 empty buffers, which is all it will ever need. The vector-based merge batcher implements this, but the logic is not simple to understand. When implementing the columnation-based merge batcher, I got it wrong, causing the seal function to temporarily retain all empty buffers, which caused an OOM situation.

We should revisit that the implementation maints the invariant that there are at most two empty buffers, and reason about why this is sufficient.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions