Tableschema registration for a table with materialized view requires the registered POJO to include those columns

[kprateek777]

Describe the bug

There are inconsistencies in how the TableSchema gets retrieved from table and from a query. This inconsistency breaks the use of the Clients client.query(select, queryParams, querySettings) API. This gets aggravated when the tableschema has some materialized columns.

Steps to reproduce

1. Create a table that has some columns that get MATERIALIZED e.g. the table has a Map named Attributes with an entry <m1, value> and m1 is materialized behind the scenes

`__m1__` LowCardinality(String) MATERIALIZED Attributes['m1'],

2. client.getTableSchema(tableName) gets the schema including the materialized columns which mandates the POJO to include even these columns. This means

client.register(MyPOjo.class, schema);

If I don't have them in the POJO then on registration I get the exception

java.lang.IllegalArgumentException: No serializer found for column '__m1__'. Did you forget to register it?

3. I should not be adding them to the POJO in the first place
4. I tried getting a schema using a SELECT query that did not have the MATERIALIZED columns which then registered fine but the client does not register the table name only the columns. Afterwards during the call to client.insert () it tries to find the table name to find out the hasDefaults and throws the following exception

            boolean hasDefaults = (Boolean)this.tableSchemaHasDefaults.get(tableName);

.
..
.
.
java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because the return value of "java.util.Map.get(Object)" is null
        at com.clickhouse.client.api.Client.insert(Client.java:1227)
        at com.clickhouse.client.api.Client.insert(Client.java:1186)

5. There is code that prevents the schema from a SELECT query to have a tableName. All of these conditions are a weird loop.

Configuration

Environment

• Client version: client v2 0.7.1-patch1 or 0.7.1
• Language version: java 17
• OS: Red Hat Enterprise Linux release 8.9 (Ootpa)

ClickHouse server

• ClickHouse Server version:
• ClickHouse Server non-default settings, if any:
• CREATE TABLE statements for tables involved:

CREATE TABLE me.me_v0
(
    `Id` UInt64 CODEC(Delta(8), ZSTD(1)),
    `Name` LowCardinality(String) CODEC(ZSTD(1)),
    `Attributes` Map(LowCardinality(String), String) CODEC(ZSTD(1)),
    `__m1__` LowCardinality(String) MATERIALIZED Attributes['m1'],
    INDEX idx_attr_key mapKeys(Attributes) TYPE bloom_filter(0.01) GRANULARITY 1,
    INDEX idx_attr_value mapValues(Attributes) TYPE bloom_filter(0.01) GRANULARITY 1
)
ENGINE = ReplicatedMergeTree('/me/tables/me_v0/{shard}', '{replica}')
ORDER BY (Name, Id)
SETTINGS index_granularity = 8192, ttl_only_drop_parts = 1

Workaround

Get the table schema via a query to exclude the materialized columns.

        selectQueryToGetSchema = String.format(selectQueryToGetSchema, tableName);

Then set the table name to the table name you want to query the client with. Set the query to null as the client.register() API mandates the schema to only have either the table name or the query

        tableSchemaFromQuery.setTableName(tableName);
        tableSchemaFromQuery.setQuery(null);

[chernser]

Good day, @kprateek777 !
Thank you for reporting the issue!