Skip to content

S3 conector does not work with Minio #59197

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 task
dedeh961 opened this issue May 2, 2025 · 1 comment
Open
1 task

S3 conector does not work with Minio #59197

dedeh961 opened this issue May 2, 2025 · 1 comment

Comments

@dedeh961
Copy link

dedeh961 commented May 2, 2025

Connector Name

airbyte/destination-s3

Connector Version

1.8.0

What step the error happened?

None

Relevant information

I'm trying to create a new s3 destination on airbyte. I'm testing my application locally with Minio.

The local structure is created in a cluster kind, so everything is created with kubernetes. Everything is created properly, and the minio and airbyte applications can connect. I tested this through http requests with curl.

Fun fact: When I use version 1.4.0 everything works perfectly. So something must have changed between versions.

Error description:

Configuration check failed

Could not connect with provided configuration. Error: software.amazon.awssdk.crt.http.HttpException: System call failure.

Relevant log output

at aws.smithy.kotlin.runtime.http.engine.crt.ConnectionManager$acquire$1.invokeSuspend(ConnectionManager.kt)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
	at kotlinx.coroutines.internal.ScopeCoroutine.afterResume(Scopes.kt:28)
	at kotlinx.coroutines.AbstractCoroutine.resumeWith(AbstractCoroutine.kt:100)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:46)
	at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:99)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:589)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:832)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:720)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:707)
Caused by: software.amazon.awssdk.crt.http.HttpException: System call failure.
	at software.amazon.awssdk.crt.http.HttpClientConnection.onConnectionAcquired(HttpClientConnection.java:130)
2025-05-02 11:11:17 info INFO main i.a.c.AirbyteConnectorRunnable(run):46 Flushing output consumer prior to shutdown.
2025-05-02 11:11:17 info INFO main i.a.c.AirbyteConnectorRunnable(run):48 Completed integration: airbyte/destination-s3.
2025-05-02 11:11:17 info Checking for optional control message...
2025-05-02 11:11:17 info Writing output of 4816b78f-1489-44c1-9060-4b19d5fa9362_e606aa68-7cdd-45e5-be22-f77b3d831fe5_0_check to the doc store
2025-05-02 11:11:18 info Marking workload 4816b78f-1489-44c1-9060-4b19d5fa9362_e606aa68-7cdd-45e5-be22-f77b3d831fe5_0_check as successful
2025-05-02 11:11:19 info 
2025-05-02 11:11:19 info Deliberately exiting process with code 0.
2025-05-02 11:11:19 info ----- END CHECK -----
2025-05-02 11:11:19 info

Contribute

  • Yes, I want to contribute
@Nestor10
Copy link

Nestor10 commented May 7, 2025

running into the exact same issues. tested each minor version from 1.5 -> 1.8 and the dev 1.8.2 version. All display the same behavior. 1.4 works. Running on a fresh kind cluster using the latest helm version of airbyte.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants