@aws-sdk/client-s3 ListObjectsV2Command hangs forever when listing many files (>800)

[moreirathomas]

What version of Bun is running?

1.0.26+c75e768a6

What platform is your computer?

Darwin 23.3.0 arm64 arm

What steps can reproduce the bug?

Sending ListObjectsCommand (or ListObjectsV2Command) via S3Client.send hangs forever at runtime.

const client = new S3Client({
  region,
  credentials: { accessKeyId, secretAccessKey },
});

const command = new ListObjectsCommand({
  Bucket: bucket,
  Prefix: prefix,
  Delimiter: "/",
});

const res = await client.send(command); // <- hangs forever

What is the expected behavior?

Do not hang forever. At least fail and return an error or throw an exception.

What do you see instead?

When testing: the test fail due to timeout.

At runtime as an http handler and called via curl: the request hangs forever (until ctrl+c).

Additional information

I have two reproduction repositories available: one running on Bun (where the bug occurs), on running on Deno (no bug reproduced).

• https://github.com/moreirathomas/aws-s3api-hangs-reproduction-bun
• https://github.com/moreirathomas/aws-s3api-hangs-reproduction-deno

Tested with @aws-sdk/client-s3 at 3.449.0 and 3.511.0.

[Electroid]

Thanks for reporting, do you happen to know if it works with smaller output sizes? or does it always hang

[moreirathomas]

It does not hang with a smaller output size (either fewer objects to be returned by S3 or pageSize is set to a smaller value, as shown in the reproduction code).

I have a S3 bucket with 2 folders, the first has 2 objects (does not hang), the second has 884 objects (hangs).

[truongan07]

RAM has increased abnormally high ?

[tusharlock10]

I am facing the same issue, bun version 1.0.28

const command = new ListObjectsV2Command({ Bucket: Env.AWS_S3_BUCKET_NAME });
const response = await client.send(command);

However in my case it hangs on the first line itself, it doesn't even reach the client.send command.

[Tirke]

For me it's all related to something like #7428 where we have similar reports. For all aws-sdk a lot of hanging is expected.

[computerdane]

This exact same issue is happening to me, but on a bucket with <200 files. Bun will hang indefinitely when trying ListObjects or ListObjectsV2. I think the breaking change was introduced somewhere in 1.2, because it works when I downgrade to 1.1.31, and breaks when I use 1.2.2. Please investigate this, my work relies on Bun for many of our data transfer jobs

[rileymiller]

I'm unable to run the ListObjectsV2 command at all using the @aws-sdk/client-s3. It just times out with a small number of files in the bucket.

[misterjame]

+1

[mkramers]

I experienced a similar issue on v1.2.5 with the following code (edited for security), however upgrading to v1.2.6 seems to resolve the issue.

import { paginateListObjectsV2, S3Client } from "@aws-sdk/client-s3";

const s3Client = new S3Client({ region: "ca-central-1" });

const BUCKET = "some_bucket";
const prefixes = ["at", "least", "four", "prefixes", "here"];

for (const prefix of prefixes) {
  const paginator = paginateListObjectsV2(
    { client: s3Client },
    { Bucket: BUCKET, Prefix: prefix },
  );

  for await (const page of paginator) {
    console.log(`Searching for objects in 's3://${BUCKET}/${prefix}'...`);
    const contents = page.Contents || [];
    console.log(`Found ${contents.length} objects`);
  }
}

# Stalls after 3 iterations of `paginator` (intermittently)
bun ./run.ts