[FIXED] Avoid filestore corruption by rejecting large publishes into JetStream #6798
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
neilalexander
changed the title
neilalexander
force-pushed
the
neil/msgrlcheck
branch
from
863ea9c to
3ce469a
Compare
neilalexander
changed the title
We have an effective and seemingly undocumented limit of 32MB for messages published into JetStream as `indexCacheBuf` in the filestore rejects any records that exceed this length. We did not catch this on publish however, and then `indexCacheBuf` would later error when consuming etc. Signed-off-by: Neil Twigg <[email protected]>
neilalexander
force-pushed
the
neil/msgrlcheck
branch
from
3ce469a to
f649114
Compare
Merged
wallyqs
changed the title
neilalexander
added a commit
that referenced
this pull request
Apr 17, 2025
Includes the following (already cherry-picked) PRs: - #6587 - #6607 - #6612 - #6609 - #6620 - #6668 - #6674 - #6647 - #6684 - #6691 - #6697 - #6705 - #6706 - #6704 - #6714 - #6720 - #6727 - #6730 - #6726 - #6732 - #6759 - #6753 - #6685 - #6769 - #6777 - #6785 - #6786 - #6778 - #6790 - #6791 - #6798 - #6794 - #6801 Signed-off-by: Neil Twigg <[email protected]> Signed-off-by: Neil Twigg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have an effective and seemingly undocumented limit of 32MB for messages published into JetStream as
indexCacheBufin the filestore rejects any records that exceed this length. We did not catch this on publish however, and thenindexCacheBufwould later error when consuming etc.Related: #6797
Signed-off-by: Neil Twigg [email protected]