[Source-S3] Invalid date-time No timezone information with Parquet and UNIX epoch

[Fabrizib]

Connector Name

source-s3

Connector Version

4.7.3

What step the error happened?

During the sync

Relevant information

I am replicating from Source-S3 a parquet file with an attribute which contains rows in the following format: 1713180693726

during replication, I can see the following traces in the logs:

ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2024-04-15T11:31:33.726000

And in the destination table, this is the format that is written: 2024-04-15T11:31:33.726000

It seems in the replication process the timezone information is dropped and it seems happening in the source S3 connector.

The attribute in the source parquet file where the timestamps are stored is of type INT64.

It seems to be a misbehaviour from the Source S3 connector that drops the timezone information when sending it into the destination writer.

Unfortunately I was not find where the "DateTimeValidator" method is for debugging this further.

Can you please help to figure out the root cause for this one?

Thanks.

Relevant log output

2024-08-14 12:01:01 ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2024-04-15T11:31:33.726000
2024-08-14 12:01:01 ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2024-04-15T11:31:33.726000
2024-08-14 12:01:01 ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2024-04-15T11:31:33.726000
2024-08-14 12:01:01 ERROR c.n.s.DateTimeValidator(tryParse):82 - Invalid date-time: No timezone information: 2024-04-15T11:31:33.726000
(...)
INFO pool-4-thread-1 i.a.i.d.t.TeradataSqlOperations(lambda$insertRecordsInternal$0):78 Airbyte Data: {"MODIFIED":"2024-04-15T11:31:33.726000","_ab_source_file_last_modified":"2024-07-11T06:46:20.000000Z","_ab_source_file_url":"folder/parequet_file.parquet"}

Contribute

• Yes, I want to contribute