Qualx unit tests

[leewyang]

This PR adds some basic unit tests for qualx.

Notes:

• Some of these are arguably integration tests, since they exercise the profiler tool, xgboost, and optuna.
• Since qualx is mostly a wrapper of these components, unit tests without these didn't seem very useful.
• As such, I had to modify tox.ini to include these dependencies. Not sure if there is a better/preferred way to do this.
• Preprocessing tests involving the Profiler tool have been moved to the E2E behave tests to pick up the compiled jar.

[amahussein]

• For the eventlogs:

  • can we put a script or a readme to show steps on how to reproduce those eventlogs?
  • For gpu-eventlog: this is important as we might need to generate the eventlog as the RAPIDS plugin evolves.

• Can we cleanup the Qualx warnings? There is an open issue for that [BUG] Reduce noise from Qualx logs #1535 . Those messages are heard to read and they pretty much show up everytime.

WARNING: Imputing missing features: ['fraction_supported', 'platform_databricks-aws', 'platform_databricks-azure', 'platform_dataproc', 'platform_emr', 'sqlOp_BatchEvalPython', 'sqlOp_CartesianProduct', 'sqlOp_CommandResult', 'sqlOp_CustomShuffleReader', 'sqlOp_DeserializeToObject', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand csv', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand json', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand orc', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand parquet', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand text', 'sqlOp_Execute InsertIntoHadoopFsRelationCommand unknown', 'sqlOp_Generate', 'sqlOp_GenerateBloomFilter', 'sqlOp_GlobalLimit', 'sqlOp_HashAggregatePrefixGroupingSets', 'sqlOp_LocalLimit', 'sqlOp_MapElements', 'sqlOp_ObjectHashAggregate', 'sqlOp_OutputAdapter', 'sqlOp_PartialWindow', 'sqlOp_ReusedSort', 'sqlOp_RunningWindowFunction', 'sqlOp_Scan ExistingRDD', 'sqlOp_Scan ExistingRDD Delta Table Checkpoint', 'sqlOp_Scan ExistingRDD Delta Table State', 'sqlOp_Scan OneRowRelation', 'sqlOp_Scan csv', 'sqlOp_Scan jdbc', 'sqlOp_Scan json', 'sqlOp_Scan orc', 'sqlOp_Scan text', 'sqlOp_Scan unknown', 'sqlOp_SerializeFromObject', 'sqlOp_SortAggregate', 'sqlOp_SubqueryOutputBroadcast', 'sqlOp_TakeOrderedAndProject', 'sqlOp_WindowSort']
WARNING: Removing extra features: ['hasSqlID', 'jobStartTime_min', 'resourceProfileId', 'sqlOp_GpuBroadcastExchange', 'sqlOp_GpuBroadcastHashJoin', 'sqlOp_GpuBroadcastNestedLoopJoin', 'sqlOp_GpuCoalesceBatches', 'sqlOp_GpuColumnarExchange', 'sqlOp_GpuColumnarToRow', 'sqlOp_GpuCustomShuff
    When preprocessing the event logs                                                # tests/spark_rapids_tools_e2e/features/steps/preprocess_steps.py:84 6.913sn parquet', 'sqlOp_GpuShuffleCoalesce', 'sqlOp_GpuShuffledHashJoin', 'sqlOp_GpuSort', 'sqlOp_GpuTopN', 'sqlOp_GpuWindow']

[leewyang]

@amahussein thanks for the review. I added a short README.md describing that these eventlogs were basically obtained from our onprem training dataset (along with the fact that they were generated via running spark-rapids-benchmarks). Also, pinned the xgboost version.

As for cleaning up the warnings, I think we should keep that separate from this PR (which is solely focused on the testing side). Feel free to assign that other bug to me, if needed. Simplest thing would be to change most of those warnings to debug logs.

[amahussein]

As for cleaning up the warnings, I think we should keep that separate from this PR (which is solely focused on the testing side). Feel free to assign that other bug to me, if needed. Simplest thing would be to change most of those warnings to debug logs.

That sounds good to me.

[amahussein]

Thanks @leewyang
LGTME