Cannot connect to DynamoDB with KCL 2.0

[wolruf]

Hello, I'm getting a report that customer cannot connect to DynamoDB using KCL 2.0 and Java SDK 2.0: "SSLEngine already closed (netty exception)"
We suspect it may be linked to Netty HTTP/2.0 not being thread safe:
netty/netty@d771526

[pfifer]

Which version of the AWS SDK, and KCL are you using? I'll ask the SDK team to take a look at this issue.

As an aside HTTP/2.0 is only used for Kinesis right now. For DynamoDB and CloudWatch the SDK uses HTTP/1.1.

[shorea]

@wolruf how often are you seeing this error?

[Nor-Wiener]

Hi All,
Thanks @wolruf for initiate this feedback on this place.
His message is related to my issue to work with the KCL.

I'm trying to stream beetwenn 50 Mbytes/s and 300 kbytes/s of SQL rows using kinesis stream.
I'm facings limits on kinesis usage restriction. (High latency / bandwith limit ).
To reduce lantency, I decided to publish group of records. The drawback of this way is that there's no more order guarantee.
In addition, I have to use multiple shards due to bandwith limit of one shard.

So, I'm trying to multiplex/demultiplex messages on kinesis shards and after that to reorder records.

I'm using the fabulous KPL for multiplexing with no issue.
I'm now stuck with the KCL.
My need is to listen on 100 streams with multithread on a EC2
I tried KCL 1 with SDK 1, I tried the KCL 2 with SDK 1 , with no success.
Then KCL 2.0 with SDK 2.

First issue I met with the last one was concerning a DynamboDB SDK 2 connection.
The exception was :
Writing request: DefaultHttpRequest(decodeResult: success, version: HTTP/1.1)
POST https://dynamodb.eu-central-1.amazonaws.com/ HTTP/1.1
amz-sdk-invocation-id: 90d3efd1-6374-1f29-b331-56bec8a97589
amz-sdk-retry: 8/150/460
Authorization: AWS4-HMAC-SHA256 Credential=AKIAJMMWXCDM42AQ5JGA/20180912/eu-central-1/dynamodb/aws4_request, SignedHeaders=amz-sdk-invocation-id;amz-sdk-retry;content-length;content-type;host;user-agent;x-amz-date;x-amz-target, Signature=b93f05db7f3e047f23db9ba6d5287ede4b4565233c5b1b8eb606d00887d4e545
Content-Length: 223
Content-Type: application/x-amz-json-1.0
Host: dynamodb.eu-central-1.amazonaws.com
User-Agent: aws-sdk-java/2.0.1 Windows_Server_2016/10.0 Java_HotSpot_TM__64-Bit_Server_VM/25.171-b11 Java/1.8.0_171
X-Amz-Date: 20180912T142307Z
X-Amz-Target: DynamoDB_20120810.CreateTable
16:23:07.910 [aws-java-sdk-NettyEventLoop-2-3] ERROR software.amazon.awssdk.http.nio.netty.internal.RunnableRequest - Failed to make request to https://dynamodb.eu-central-1.amazonaws.com/
javax.net.ssl.SSLException: SSLEngine closed already
at io.netty.handler.ssl.SslHandler.wrap(...)(Unknown Source)
Done!
software.amazon.awssdk.core.exception.SdkClientException

So I decided to not use the KCL but only the SDK 2.0.
I tried to run this exemple : https://docs.aws.amazon.com/fr_fr/streams/latest/dev/building-enhanced-consumers-api.html

It works with success on One thread but It appears, it's not thread Safe due to netty http2 bug :netty/netty@d771526

I will publish my source code this afternoon.