-
Notifications
You must be signed in to change notification settings - Fork 2k
[BugFix] fix wrong order by scope for distinct query #37910
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
allFields.add(field); | ||
} | ||
return allFields; | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The most risky bug in this code is:
Inconsistent behavior when parsing ORDER BY expression for queries with SELECT DISTINCT.
You can modify the code like this:
// In analyzeOrderBy method, adjust the part where aggregations are checked within a SELECT DISTINCT context
if (isDistinct && !aggregations.isEmpty()) {
// Here we need to not only throw an exception if there are aggregations,
// but also ensure that columns in the ORDER BY clause are present in the SELECT list.
boolean orderByNotInSelectList = !outputExpressions.containsAll(expression.toExprList());
if(orderByNotInSelectList) {
throw new SemanticException("for SELECT DISTINCT, ORDER BY expressions must appear in the select list",
expression.getPos());
}
}
// Later in the same method, we also need to ensure that the ORDER BY fields are consistently validated
// against the correct scope when SELECT DISTINCT is used.
if (!isDistinct) {
analyzeExpression(expression, analyzeState, orderByScope);
} else {
analyzeExpression(expression, analyzeState, orderByScope.getParent());
}
This addresses the issue where the ORDER BY clause might reference expressions not in the SELECT list when using DISTINCT, which should result in an error as implemented in this proposed fix. Additionally, when SELECT DISTINCT is true, the ORDER BY expressions would correctly be analyzed against the parent scope rather than the possibly incorrect orderByScope.
Signed-off-by: packy92 <[email protected]>
Signed-off-by: packy92 <[email protected]>
Signed-off-by: packy92 <[email protected]>
726a50c
to
5eb931e
Compare
Signed-off-by: packy92 <[email protected]>
|
[FE Incremental Coverage Report]✅ pass : 36 / 44 (81.82%) file detail
|
[BE Incremental Coverage Report]✅ pass : 0 / 0 (0%) |
@Mergifyio backport branch-3.2 |
@Mergifyio backport branch-3.1 |
@Mergifyio backport branch-3.0 |
✅ Backports have been created
|
@Mergifyio backport branch-2.5 |
✅ Backports have been created
|
✅ Backports have been created
|
✅ Backports have been created
|
Signed-off-by: packy92 <[email protected]> (cherry picked from commit fa72214) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/sql/analyzer/AggregationAnalyzer.java
Signed-off-by: packy92 <[email protected]> (cherry picked from commit fa72214) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/SessionVariable.java # fe/fe-core/src/main/java/com/starrocks/sql/analyzer/AggregationAnalyzer.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/OrderByTest.java
Signed-off-by: packy92 <[email protected]> (cherry picked from commit fa72214) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/SessionVariable.java # fe/fe-core/src/main/java/com/starrocks/sql/analyzer/AggregationAnalyzer.java # fe/fe-core/src/test/java/com/starrocks/sql/parser/ParserTest.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/OrderByTest.java
Signed-off-by: packy92 <[email protected]> (cherry picked from commit fa72214) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/qe/SessionVariable.java # fe/fe-core/src/main/java/com/starrocks/sql/analyzer/AggregationAnalyzer.java # fe/fe-core/src/test/java/com/starrocks/sql/parser/ParserTest.java # fe/fe-core/src/test/java/com/starrocks/sql/plan/OrderByTest.java
…cks#37910) Signed-off-by: packy92 <[email protected]>
…cks#37910) Signed-off-by: packy92 <[email protected]>
#38064) Signed-off-by: packy92 <[email protected]>
…cks#37910) Signed-off-by: packy92 <[email protected]>
#38057) Signed-off-by: packy92 <[email protected]>
…cks#37910) Signed-off-by: packy92 <[email protected]>
#38085) Signed-off-by: packy92 <[email protected]>
#38086) Signed-off-by: packy92 <[email protected]>
Why I'm doing:
Fail to execute
select distinct * from tbl order by col
.Root Casue:
The scope for order by is like `scope([], parentScope(tbl.col1, tbl.col2, ...)), so the analyze result of col is a field from parent scope filed. When check whether the col is a valid expr for aggregation expr, the col should be in the original order by scope instead of its parent scope. If not, error happens.
Actually, the scope for order should be the output of the distinct and cannot 'see' its parent scope.
What I'm doing:
enable_strict_order_by
session variable default value is true to keep same behavior in 2.3 version.set enable_strict_order_by = false
to execute sql likeselect distinct *, abs(v1) v1 from tbl order by v1
which is a valid sql in MySQL.Fixes #issue
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check: