[FEA] Qual tool should evaluate Hive formats support

[amahussein]

Is your feature request related to a problem? Please describe.

The RAPIDS plugin added support to insertIntoHiveTable NVIDIA/spark-rapids#10912 . However, the dataSource.csv did not add a new entry to indicate that.

According to NVIDIA/spark-rapids#10912, the HiveParquet is supported when:

• parquetOutputFormatClass = "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat"
• parquetSerdeClass = "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe"
• table is not bucketed

The problems:

• InsertHiveTable in the eventlog shows the serDe library but I don't see the outputFormatClass.
• How to find whether a table is bucketed or not ?
• the qual prediction might be off if the user disable spark.sql.hive.convertMetastoreParquet on the GPU. Even if we recommend turning that on, there is no guarantee users can do it due to compatibility issues.

Other Formats:

I also see that HiveORC is converted into GPU. We might need to change the format to be "ORC" when the Spark configurations use native ORC.

InsertIntoHadoopFsRelationCommand HiveORC
# this will be transformed into 
GpuInsertIntoHadoopFsRelationCommand HiveORC

I kept eventlog representing that case in my scratch-folder under the same issue number (app-20250305030545-0000.zstd).

Related issues

#1583

[amahussein]

See related bug filed on RAPIDS plugin Repo
blocked by NVIDIA/spark-rapids#12438