fix(query): improve update handling about nondeterministic_update and add nullable columns

[loloxwg]

Added replace_column_datatype_to_nullable method to support updates involving nullable column types. Refactored rewrite_nondeterministic_update logic to use matcher-based pattern matching for enhanced clarity and maintainability. Updated tests to include scenarios with CTEs and nullable columns for improved test coverage.

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

Tests

• Unit Test
• Logic Test
• Benchmark Test
• No Test - Explain why

Type of change

• Bug Fix (non-breaking change which fixes an issue)
• New Feature (non-breaking change which adds functionality)
• Breaking Change (fix or feature that could cause existing functionality not to work as expected)
• Documentation Update
• Refactoring
• Performance Improvement
• Other (please describe):

---

This change is 

[BohuTANG]

Thanks for contributing!
The PR is failing tests - please check CI logs, and let us know if need assistance.

[forsaken628]

Please provide some test cases to reproduce the bug you wish to fix

[loloxwg]

@forsaken628 #17573

[loloxwg]

Thanks for contributing! The PR is failing tests - please check CI logs, and let us know if need assistance.

Should the default field type of a temporary table created with a WITH statement be set to NULL? I believe doing so would resolve the issue. @BohuTANG @forsaken628

[forsaken628]

@loloxwg

-- This sql is working fine.
with tbb as (select col0::string null col1,col1::string null col2,col2::string null col3 from (values ('1', 'add', '11'), ('4', 'add', '44')))
update test_merge tba set tba.col1 =tbb.col1, tba.col2 = 'update', tba.col3 = tbb.col3 from tbb where tba.col1 = tbb.col1;


-- But this one no work
with tbb("col1", "col2", "col3") as (values ('1', 'add', '11'), ('4', 'add', '44'))
update test_merge tba set tba.col1 =tbb.col1::string null, tba.col2 = 'update', tba.col3 = tbb.col3::string null from tbb where tba.col1 = tbb.col1::string null;

My guess is that type_check is not handling the type correctly, so you can check it out.

[loloxwg]

@loloxwg

-- This sql is working fine.
with tbb as (select col0::string null col1,col1::string null col2,col2::string null col3 from (values ('1', 'add', '11'), ('4', 'add', '44')))
update test_merge tba set tba.col1 =tbb.col1, tba.col2 = 'update', tba.col3 = tbb.col3 from tbb where tba.col1 = tbb.col1;


-- But this one no work
with tbb("col1", "col2", "col3") as (values ('1', 'add', '11'), ('4', 'add', '44'))
update test_merge tba set tba.col1 =tbb.col1::string null, tba.col2 = 'update', tba.col3 = tbb.col3::string null from tbb where tba.col1 = tbb.col1::string null;

My guess is that type_check is not handling the type correctly, so you can check it out.

I fixed update stmt above by this way, but i import new issue, cant insert literal string value
insert into test_merge values(2,'abc',2),(3,'abc',3),(4,'abc',4);

[forsaken628]

databend/src/query/sql/src/planner/binder/bind_mutation/update.rs

Lines 169 to 180 in cd95677

	let used_columns = mutation
	.matched_evaluators
	.iter()
	.flat_map(|eval| {
	eval.update.iter().flat_map(|update| {
	update
	.values()
	.flat_map(|expr| expr.used_columns().into_iter())
	})
	})
	.chain(mutation.required_columns.iter().copied())
	.collect::<HashSet<_>>();

If expr is a not null type, cast to a nullable type, because the following aggregation will converts the column to nullable

[loloxwg]

databend/src/query/sql/src/planner/binder/bind_mutation/update.rs

Lines 169 to 180 in cd95677

	let used_columns = mutation
	.matched_evaluators
	.iter()
	.flat_map(|eval| {
	eval.update.iter().flat_map(|update| {
	update
	.values()
	.flat_map(|expr| expr.used_columns().into_iter())
	})
	})
	.chain(mutation.required_columns.iter().copied())
	.collect::<HashSet<_>>();

If expr is a not null type, cast to a nullable type, because the following aggregation will converts the column to nullable

how can i change the block datatype

[drmingdrmer]

The nesting level is too deep and is hard to understand. Please extract inner logic to some small function or use intermediate variable to improve the readability.

[forsaken628]

how can i change the block datatype

here 0638e0f

Next you need to change the PR title and add some tests.
Should have a look if there are some other tests which will fail if error_on_nondeterministic_update = 1.

[loloxwg]

how can i change the block datatype

Add a RelOperator::EvalScalar between RelOperator::Mutation and RelOperator::Aggregate then cast nullable to not null.

If I haven't misunderstood the meaning, should it be like this? And how can i fill the EvaScalar' s items?

[loloxwg]

why ci test failed?

[sundy-li]

maybe also need change to:

let mut aggr_func = ScalarExpr::AggregateFunction(AggregateFunction {
                    span: None,
                    func_name: "any".to_string(),
                    distinct: false,
                    params: vec![],
                    args: vec![ScalarExpr::BoundColumnRef(BoundColumnRef {
                        span: None,
                        column: binding.clone(),
                    })],
                    return_type: Box::new(binding.data_type.clone().wrap_nullable()),
                    sort_descs: vec![],
                    display_name: display_name.clone(),
                });

🐳 root@default:)  with tbb("col1", "col2", "col3") as (values ('1', 'add', '11'), ('4', 'add', '44'))
update test_merge tba set tba.col1 =tbb.col1, tba.col2 = 'update', tba.col3 = tbb.col3 from tbb where tba.col1 = tbb.col1;


╭────────────────────────╮
│ number of rows updated │
│         UInt64         │
├────────────────────────┤
│                      1 │
╰────────────────────────╯
1 row read in 0.343 sec. Processed 3 row, 133 B (8.75 rows/s, 387 B/s)