[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' #56565

click9000 · 2025-03-04T17:59:57Z

Why I'm doing:

For sql_dialect='trino' case, parse_datetime('2025-01-01T00:00:00.000Z', 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z''') should be transformed

What I'm doing:

Replace 'T' to T and remove 'Z' in the format string for function parse_datetime()

Example:
parse_datetime('2025-01-01T00:00:00.000Z', 'yyyy-MM-dd''T''HH:mm:ss.SSS''Z''')

should be converted to

str_to_jodatime('2025-01-01T00:00:00.000Z',, 'yyyy-MM-ddTHH:mm:ss.SSS')

Fixes #issue

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
This is a backport pr

Bugfix cherry-pick branch check:

for function from_iso8601_timestamp -> timestamp Signed-off-by: Craig <[email protected]>

unit test for transforming Trino function: from_iso8601_timestamp -> timestamp Signed-off-by: Craig <[email protected]>

Signed-off-by: Craig <[email protected]>

move function parse_datetime() transform to ComplexFunctionCallTransformer.java to process format string when needed Signed-off-by: Craig <[email protected]>

Move function parse_datetime() transform to ComplexFunctionCallTransformer.java Signed-off-by: Craig <[email protected]>

add a test case for str_to_jodatime() transform Signed-off-by: Craig <[email protected]>

Signed-off-by: Craig <[email protected]>

merge the if statement per SonarQube Cloud's check Signed-off-by: Craig <[email protected]>

remove case for parse_datetime: 'yyyy-MM-dd''T''HH:mm:ss''Z''' Signed-off-by: Craig <[email protected]>

Update per SonarQube check Signed-off-by: Craig <[email protected]>

stephen-shelby · 2025-03-06T12:51:18Z

@Youngwb please take a look

Youngwb · 2025-03-07T01:59:02Z

...-core/src/main/java/com/starrocks/connector/parser/trino/ComplexFunctionCallTransformer.java

+            // parse_datetime -> str_to_jodatime
+            String formatString = ((StringLiteral) args[1]).getStringValue();
+            // "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'" -> "yyyy-MM-ddTHH:mm:ss.SSS"
+            formatString = formatString.replace("'T'", "T").replace("'Z'", "");


I think you should handle any character within the quotes, rather than just handling the character 'T'.
eg. select parse_datetime('2025-01-01ABC00:00:00.000Z', 'yyyy-MM-dd''ABC''HH:mm:ss.SSS''Z''');

I can remove all ' characters for the format parameter, plus removing all Z characters because str_to_jodatime doesn't support it (when it's OK in Trino).
current example with Z in the format parameter:

mysql> set sql_dialect=trino; mysql> select parse_datetime('2023-08-02T14:37:02Z', 'yyyy-MM-dd''T''HH:mm:ssZ'); ERROR 1064 (HY000): Invalid format 'yyyy-MM-ddTHH:mm:ssZ' for '2023-08-02T14:37:02Z' mysql> select parse_datetime('2023-08-02T14:37:02Z', 'yyyy-MM-dd''T''HH:mm:ss''Z'''); +----------------------------------------------------------------+ | str_to_jodatime('2023-08-02T14:37:02Z', 'yyyy-MM-ddTHH:mm:ss') | +----------------------------------------------------------------+ | 2023-08-02 14:37:02 | +----------------------------------------------------------------+

code updated and add one more test for function parse_datetime()

Youngwb · 2025-03-07T01:59:18Z

@click9000 Could you plz add some test to cover this case?

Add one more test for function parse_datetime() to handle format case like 'yyyy-MM-dd''T''HH:mm:ss''Z''' Signed-off-by: Craig <[email protected]>

Remove all ' characters for the format parameter, plus removing all Z characters because str_to_jodatime doesn't support it (when it's OK in Trino) Signed-off-by: Craig <[email protected]>

sonarqubecloud · 2025-03-09T07:00:31Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions · 2025-03-09T08:31:30Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-03-09T08:32:04Z

[FE Incremental Coverage Report]

✅ pass : 4 / 4 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/connector/parser/trino/ComplexFunctionCallTransformer.java	4	4	100.00%	[]

github-actions · 2025-03-09T08:32:24Z

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2025-03-10T17:42:46Z

@Mergifyio backport branch-3.4

github-actions · 2025-03-10T17:42:46Z

@Mergifyio backport branch-3.3

mergify · 2025-03-10T17:42:56Z

backport branch-3.4

✅ Backports have been created

#56789 [Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) has been created for branch branch-3.4

mergify · 2025-03-10T17:42:58Z

backport branch-3.3

✅ Backports have been created

#56790 [Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) has been created for branch branch-3.3

…time(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (#56565) Signed-off-by: Craig <[email protected]> (cherry picked from commit b8e05fe)

…time(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) (#56789) Co-authored-by: Craig <[email protected]>

…time(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) (#56790) Co-authored-by: Craig <[email protected]>

click9000 added 11 commits February 5, 2025 11:13

Update Trino2SRFunctionCallTransformer.java

cfeeb92

for function from_iso8601_timestamp -> timestamp Signed-off-by: Craig <[email protected]>

Update TrinoFunctionTransformTest.java

b52e093

unit test for transforming Trino function: from_iso8601_timestamp -> timestamp Signed-off-by: Craig <[email protected]>

Update TrinoFunctionTransformTest.java

d5a0d16

Signed-off-by: Craig <[email protected]>

Update TrinoFunctionTransformTest.java

2edae11

Signed-off-by: Craig <[email protected]>

Merge branch 'StarRocks:main' into main

239ece5

Merge branch 'StarRocks:main' into main

11eabef

Update Trino2SRFunctionCallTransformer.java

7bbafc8

move function parse_datetime() transform to ComplexFunctionCallTransformer.java to process format string when needed Signed-off-by: Craig <[email protected]>

Update ComplexFunctionCallTransformer.java

0bfcf30

Move function parse_datetime() transform to ComplexFunctionCallTransformer.java Signed-off-by: Craig <[email protected]>

Update TrinoFunctionTransformTest.java

a2f7df8

add a test case for str_to_jodatime() transform Signed-off-by: Craig <[email protected]>

Update TrinoFunctionTransformTest.java

8aebea1

Signed-off-by: Craig <[email protected]>

Merge branch 'StarRocks:main' into main

d46de77

click9000 requested a review from a team as a code owner March 4, 2025 17:59

github-actions bot added behavior_changed title needs [type] labels Mar 4, 2025

mergify bot assigned click9000 Mar 4, 2025

Update ComplexFunctionCallTransformer.java

01fad40

merge the if statement per SonarQube Cloud's check Signed-off-by: Craig <[email protected]>

github-actions bot added 3.3 3.4 and removed title needs [type] labels Mar 4, 2025

click9000 added 2 commits March 4, 2025 13:28

Update TrinoFunctionTransformTest.java

9159145

remove case for parse_datetime: 'yyyy-MM-dd''T''HH:mm:ss''Z''' Signed-off-by: Craig <[email protected]>

Update ComplexFunctionCallTransformer.java

1fb5874

Update per SonarQube check Signed-off-by: Craig <[email protected]>

Youngwb reviewed Mar 7, 2025

View reviewed changes

click9000 added 3 commits March 8, 2025 22:19

Merge branch 'StarRocks:main' into main

541ce15

Update TrinoFunctionTransformTest.java

e37f8e4

Add one more test for function parse_datetime() to handle format case like 'yyyy-MM-dd''T''HH:mm:ss''Z''' Signed-off-by: Craig <[email protected]>

Update ComplexFunctionCallTransformer.java

b5db10f

Remove all ' characters for the format parameter, plus removing all Z characters because str_to_jodatime doesn't support it (when it's OK in Trino) Signed-off-by: Craig <[email protected]>

Youngwb approved these changes Mar 10, 2025

View reviewed changes

Youngwb enabled auto-merge (squash) March 10, 2025 05:44

dirtysalt approved these changes Mar 10, 2025

View reviewed changes

Youngwb merged commit b8e05fe into StarRocks:main Mar 10, 2025
54 checks passed

github-actions bot removed 3.4 3.3 labels Mar 10, 2025

mergify bot mentioned this pull request Mar 10, 2025

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) #56789

Merged

18 tasks

mergify bot mentioned this pull request Mar 10, 2025

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' (backport #56565) #56790

Merged

18 tasks

github-actions bot added the 3.4-merged label Mar 19, 2025

github-actions bot added the 3.3-merged label Mar 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' #56565

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' #56565

Uh oh!

click9000 commented Mar 4, 2025

Uh oh!

stephen-shelby commented Mar 6, 2025

Uh oh!

Youngwb Mar 7, 2025

Uh oh!

click9000 Mar 9, 2025

Uh oh!

click9000 Mar 9, 2025

Uh oh!

Youngwb commented Mar 7, 2025

Uh oh!

sonarqubecloud bot commented Mar 9, 2025

Uh oh!

github-actions bot commented Mar 9, 2025

Uh oh!

github-actions bot commented Mar 9, 2025

Uh oh!

github-actions bot commented Mar 9, 2025

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

mergify bot commented Mar 10, 2025 •

edited

Loading

Uh oh!

mergify bot commented Mar 10, 2025 •

edited

Loading

Uh oh!

Uh oh!

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' #56565

[Enhancement] Convert sql function parse_datetime(...) to str_to_jodatime(...) with accepting 'T' and 'Z' in format for Trino transformer when sql_dialect='trino' #56565

Uh oh!

Conversation

click9000 commented Mar 4, 2025

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

Uh oh!

stephen-shelby commented Mar 6, 2025

Uh oh!

Youngwb Mar 7, 2025

Choose a reason for hiding this comment

Uh oh!

click9000 Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

click9000 Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

Youngwb commented Mar 7, 2025

Uh oh!

sonarqubecloud bot commented Mar 9, 2025

Quality Gate passed

Uh oh!

github-actions bot commented Mar 9, 2025

[Java-Extensions Incremental Coverage Report]

Uh oh!

github-actions bot commented Mar 9, 2025

[FE Incremental Coverage Report]

file detail

Uh oh!

github-actions bot commented Mar 9, 2025

[BE Incremental Coverage Report]

Uh oh!

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

github-actions bot commented Mar 10, 2025

Uh oh!

mergify bot commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

mergify bot commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Uh oh!

mergify bot commented Mar 10, 2025 •

edited

Loading

mergify bot commented Mar 10, 2025 •

edited

Loading