feat(frontend): add settings keyword for runtime parameter

[zwang28]

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR adds the SETTINGS keyword for Statement::Query, to allow a query to overwrite its session runtime parameters.

The core changes are minor in terms of lines of code (LOC). Unfortunately I have to change all usage of session.config().xxx() to running_sql_runtime_parameters(RuntimeParameters::xxx) in order to expose RuntimeParameters inside SessionImpl::running_sql_runtime_parameters of ArcSwap<RwLock> type, which increases changes in LOC.

#21218

https://risingwave-labs.slack.com/archives/C034U0NH5ND/p1744175012756489

Checklist

• I have written necessary rustdoc comments.
• I have added necessary unit tests and integration tests.
• I have added test labels as necessary.
• I have added fuzzing tests or opened an issue to track them.
• My PR contains breaking changes.
• My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
• I have checked the Release Timeline and Currently Supported Versions to determine which release branches I need to cherry-pick this PR into.

Documentation

• My PR needs documentation updates.

Release note

[github-actions]

Hi, there.

📝 Telemetry Reminder:
If you're implementing this feature, please consider adding telemetry metrics to track its usage. This helps us understand how the feature is being used and improve it further.
You can find the function report_event of telemetry reporting in the following files. Feel free to ask questions if you need any guidance!

• src/frontend/src/telemetry.rs
• src/meta/src/telemetry.rs
• src/stream/src/telemetry.rs
• src/storage/compactor/src/telemetry.rs
  Or calling report_event_common (src/common/telemetry_event/src/lib.rs) as if finding it hard to implement.
  ✨ Thank you for your contribution to RisingWave! ✨

---

This is an automated comment created by the peaceiris/actions-label-commenter. Responding to the bot or mentioning it won't have any effect.

[chenzl25]

Can we use the same syntax as pg_hint_plan? That is supporting SQL hint.
For example:

/*+ Set(query_mode local) */ SELECT * FROM table1 t1 WHERE key = 'value';

Once we have the SQL hint, we can extend it to support more features, e.g., changing the query plan.

https://github.com/ossc-db/pg_hint_plan/blob/master/docs/hint_table.md#guc-parameters-set-during-planning

[chenzl25]

Related issue #7491

[BugenZhao]

This somehow reminds me of...

• discussion: shall we introduce WITH properties for materialized views? #11929 (comment)
• store "pure" SQL in the Source/Table's definition #17472
• feat: support DESCRIBE FRAGMENTS #21354

In short, we may consider persisting all session variables referenced by the optimizer into the job definition to achieve better reproducibility when replanning it. The introduction of the SETTINGS clause provides a natural location to put them.

[xiangjinwu]

Reserving any new keyword is a breaking change. If a user has existing mviews / tables / sources / columns named settings, their definition stored in meta catalog would become invalid.

In an old version, a user can:

create table t (settings struct<key varchar, value varchar>[]);

In the new version, if we do

alter table t add column foo int;

It would result in:

ERROR:  Failed to run the query

Caused by these errors (recent errors listed first):
  1: unable to parse definition sql
  2: sql parser error: syntax error at or near settings

Because to create the same table, the correct SQL in new version is:

create table t ("settings" struct<key varchar, value varchar>[]);

[zwang28]

Would you recommend customizing the parser part to avoid reserving new keywords?

[xiangjinwu]

Just something on top of my mind: one of the the least intrusive ways to extend the Query syntax is probably the WITH cte clause at the beginning.

In addition to recognizing either the RECURSIVE keyword or a non-reserved identifier as cte_name, we could peek for another already reserved keyword (e.g. USING or another word that reads better) for this extended settings clause. For example:

WITH USING (search_path=schema_2 ,query_mode=auto)
WITH foo AS (select 1), bar AS (select 2)
SELECT * FROM foo, bar;

risingwave/src/sqlparser/src/parser.rs

Lines 4419 to 4426 in 017748e

	let with = if self.parse_keyword(Keyword::WITH) {
	Some(With {
	recursive: self.parse_keyword(Keyword::RECURSIVE),
	cte_tables: self.parse_comma_separated(Parser::parse_cte)?,
	})
	} else {
	None
	};

update: WITH CONSTRAINT is more natural than WITH USING grammarly. But CONSTRAINT has a concrete meaning in DBs while USING is a generic word.

Alternatively, @bugen is likely suggesting a trailing WITH for consistency with existing mview / sink syntax:

SELECT * FROM foo WITH (...);
CREATE MATERIALIZED VIEW foo WITH (...) AS query; -- existing syntax, also in PostgreSQL
CREATE SINK foo AS query WITH (...); -- existing syntax

However, consistency can also mean ambiguity here. If we add a trailing WITH as part of Query, it would result in:

CREATE MATERIALIZED VIEW foo WITH (..mview..) AS SELECT * FROM foo WITH (..query..);
CREATE SINK foo AS SELECT * FROM foo WITH (..query..) WITH (..sink..);

The latter cast is ambiguous when the WITH-of-query is omitted. The parse_query function may consume the WITH (..sink..) part mistakenly.

The former case, as well as the leading WITH USING syntax proposed above, reads a little bit unfriendly (which parameter should I write in which parentheses) but unambiguous - thanks to the AS keyword and existing support of WITH-cte.

CREATE MATERIALIZED VIEW foo WITH (...) AS WITH USING (...) SELECT * FROM foo;
CREATE SINK foo AS WITH USING (...) SELECT * FROM foo WITH (...);

Further iterations on this rough idea are welcomed @BugenZhao @chenzl25

[chenzl25]

How about using the PostgreSQl syntax for materialized view https://www.postgresql.org/docs/current/sql-creatematerializedview.html like:

create materialized view v with(xxx = yyy) as select * from t;
create materialized view mv with(xxx = yyy) as with cte as (select 1, 2, 3) select * from cte;

[xiangjinwu]

The materialized view syntax is already supported. The discussion here is about the syntax when it is not a materialized view - and whether it should look like the existing materialized view syntax

[chenzl25]

Considering no matter for batch queries or streaming queries, we want to capture the session variables/parameters, and we don't want to introduce a new keyword which would cause a breaking change, beside that using with keyword is a bit confused to users since with options are in the different position of our MV and sink syntax, furthermore, batch queries don't support with options. I think maybe we can just use using session parameter (...) to capture the session parameters to declare the purpose precisely without ambiguity.

Example:

select * from t using session parameter (xxx = yyy);
create materialized view v as select * from t using session parameter (xxx = yyy);
create materialized view mv as with cte as (select 1, 2, 3) select * from cte using session parameter (xxx = yyy);

[chenzl25]

I just realized that SETTINGS is a keyword of clickhouse, and they have a session setting syntax similar to this PR, so I think we can use SETTINGS as a keyword as well.
https://clickhouse.com/docs/operations/settings/settings

[BugenZhao]

Shall we follow the syntax of WITH clause for better consistency? Kindly ask @xiangjinwu for insights.

[github-actions]

This PR has been open for 60 days with no activity.

If it's blocked by code review, feel free to ping a reviewer or ask someone else to review it.

If you think it is still relevant today, and have time to work on it in the near future, you can comment to update the status, or just manually remove the no-pr-activity label.

You can also confidently close this PR to keep our backlog clean. (If no further action taken, the PR will be automatically closed after 7 days. Sorry! 🙏)
Don't worry if you think the PR is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄