JetStream: Messages >32MB cause corrupt index and malformed/corrupt error, despite server limits allowing it [v2.10.27]

[wartis]

Observed behavior

When sending a message of exactly 32MB or larger using .jetStream().publish, I get the following server-side error during message lookup:

[WRN] Filestore [STR_1] indexCacheBuf corrupt record state: dlen 33554452 slen 12 index 30 rl 33554474 lbuf 33554504
[ERR] Received an error looking up message for consumer '$G > TEST_STREAM > TEST_STREAM_DURABLE': malformed or corrupt message

The consumer cannot receive the message, and JetStream behaves as if the message index is corrupted. The issue occurs consistently for payloads ≥ 32MB, even though they're below max_payload.
The same code and message size work correctly in NATS v2.2, suggesting a regression in later versions.

Expected behavior

Messages up to max_payload (64MB) should be accepted and correctly stored/delivered by JetStream.

Server and client version

• NATS Server version (failing): 2.11, 2.10

• NATS Server version (working): 2.2.x

• Client: Java (also reproducible with Go)

• Client Library Version: nats.java 2.21, 2.20, 2.11

Host environment

Environment:

• Deployment: Docker

• OS: Ubuntu 24.04.2 LTS

---

JetStream Configuration:

jetstream {
  max_mem_store: 4GB
  max_file_store: 100GB
  store_dir: /data/jetstream
}

Limits:

max_payload: 64MB
max_pending: 64MB

Steps to reproduce

Below is a simplified version of the code I’m using to send random byte arrays as test messages:

public void sendRandomBytesRequest(String subject, int lengthInMb) throws JetStreamApiException, IOException {
    int lengthInBytes = lengthInMb * 1024 * 1024;
    byte[] data = new byte[lengthInBytes];
    new Random().nextBytes(data);

    sendRequest(subject, data);
}

public void sendRequest(String subject, byte[] data) throws JetStreamApiException, IOException {
    NatsMessage message = NatsMessage.builder().subject(subject).data(data).build();
    connection.jetStream().publish(message, createDefaultPublisherOptions(streamName).build());
}

Demo method:

sender.sendRandomBytesRequest(subject, 10); // works
sender.sendRandomBytesRequest(subject, 20); // works
sender.sendRandomBytesRequest(subject, 30); // works
sender.sendRandomBytesRequest(subject, 32); // fails

Also fails when:

• Trying to publish a large file (>32MB) as byte[]

• Same result using file-backed storage or memory

[wartis]

Notes:

• This appears to be a regression introduced after NATS 2.2

• The problem affects both memory and file-backed JetStream storage

• There is no client-side exception on publish

• The same message sizes work correctly in NATS 2.2, including 32MB+

• Java client calls JetStream.publish(...) with a valid message

• The link to the test repo: https://github.com/wartis/JNatsJetStreamError/tree/nats-2.11

---

Request:

• Is there a new undocumented limit or storage change since v2.2?

• Could this be a known issue or regression related to message chunking/indexing?

Let me know if logs or more details would be helpful!

Thanks!

[neilalexander]

There is indeed an effective limit of 32MB here, although it would appear it has not been documented or properly enforced. The filestore isn't really designed to accommodate messages larger than this (can run into performance problems with the block indexing) so we should instead add proper error handling to detect this case at publish time.

Can I ask what the use-case is for such large messages? Would it be met instead by using object store?

[wartis]

Thanks for the quick response and clarification!

Our use case is that we use NATS as the primary communication channel between our microservices. In some cases, the services exchange .eml files (emails), which can sometimes be quite large—up to 38MB in our experience.

We also use nats.js on the frontend, and large files like emails are occasionally passed back and forth between the frontend and backend.

So, using an Object Store might be an alternative. We're currently considering two approaches:

1. Uploading the data to an Object Store and sending metadata (like where the data is stored in the object store) via streams.
2. Chunking the data.