Destination S3: Support Apache Iceberg file format

[sherifnada]

Tell us about the problem you're trying to solve

Apache Iceberg is a popular format for database engines like Trino, Spark, Hive, and others. it is strongly typed and specifically suited for handling schema evolution because it keeps track of a schema across many files (whereas Parquet/etc.. keep track of schema at the file level). It would be valuable to support Iceberg in the S3 destination as it unlocks better support for databases like Trino.

Describe the solution you’d like

Iceberg support for the S3 destination connector

Describe the alternative you’ve considered or used

Parquet with a file-based Hive metastore

[gsmith-schlesinger]

Are there places where airbyte-iceberg-s3 discussions are happening? Or is this the place? Is there a fork where this effort is currently being carried forward?

[bitsondatadev]

This looks completed, @Leibnizhu / @grishick, should this get closed?

[Leibnizhu]

This looks completed, @Leibnizhu / @grishick, should this get closed?

i think so

[jankap]

What about normalization in iceberg target? Is it only possible to write JSON?

[octavia-squidington-iii]

At Airbyte, we seek to be clear about the project priorities and roadmap. This issue has not had any activity for 365 days, suggesting that it's not as critical as others. It's possible it has already been fixed. It is being marked as stale and will be closed in 20 days if there is no activity. To keep it open, please comment to let us know why it is important to you and if it is still reproducible on recent versions of Airbyte.