Skip to content

Fixes parser for the top level tokens #369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 8, 2021
Merged

Fixes parser for the top level tokens #369

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7d7e2e9
Fixed parser for the top level tokens
abhikuhikar Feb 25, 2021
caebc9f
Small changes
abhikuhikar Feb 26, 2021
1a8ed3d
Added more tests and more top level checks
abhikuhikar Mar 2, 2021
fec50de
Updated algorithm to include DDL parse types
abhikuhikar Mar 4, 2021
a9de025
Added logic to handle top level DML tokens
abhikuhikar Mar 5, 2021
f375d2d
Addressed comments
abhikuhikar Mar 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions lang/src/org/partiql/lang/syntax/SqlParser.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2254,19 +2254,19 @@ class SqlParser(private val ion: IonSystem) : Parser {
}

private fun ParseNode.throwTopLevelParserError(): Nothing =
token?.err("Keyword $token only expected at the top level in the query", PARSE_UNEXPECTED_TERM)
?: throw ParserException("Keyword $token only expected at the top level in the query", PARSE_UNEXPECTED_TERM, PropertyValueMap())
token?.err("Keyword ${token.text} only expected at the top level in the query", PARSE_UNEXPECTED_TERM)
?: throw ParserException("Keyword ${token?.text} only expected at the top level in the query", PARSE_UNEXPECTED_TERM, PropertyValueMap())

/**
* Validates tree to make sure that the top level tokens are not found below the top level
* Validates tree to make sure that the top level tokens are not found below the top level.
* Top level tokens are the tokens or keywords which are valid to be used only at the top level in the query.
* i.e. these tokens cannot be used with a mix of other commands. Hence if more than one top level tokens are found
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "i.e. these tokens cannot be used with a mix of other commands top level tokens"? "commands" seems too generalized.

* in the query then it is invalid.
* [level] is the current traversal level in the parse tree.
* If [topLevelTokenSeen] is true, it means it has been encountered at least once before while traversing the parse tree.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: description of topLevelTokenSeen could be more more specific. Consider "...it means a top level token has been previously encountered while traversing the parse tree"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the variable name itself tells it all.

*/
private fun validateTopLevelNodes(node: ParseNode, level: Int, topLevelTokens: Int) {
val topTokens = topLevelTokens + when(node.type.isTopLevelType) {
true -> 1
false -> 0
}

if (topTokens > 1) {
private fun validateTopLevelNodes(node: ParseNode, level: Int, topLevelTokenSeen: Boolean) {
if (topLevelTokenSeen && node.type.isTopLevelType) {
node.throwTopLevelParserError()
}

Expand All @@ -2280,7 +2280,7 @@ class SqlParser(private val ion: IonSystem) : Parser {
node.throwTopLevelParserError()
}
}
node.children.map { validateTopLevelNodes(it, level + 1, topTokens) }
node.children.map { validateTopLevelNodes(node = it, level = level + 1, topLevelTokenSeen = topLevelTokenSeen || node.type.isTopLevelType) }
}

/** Entry point into the parser. */
Expand All @@ -2296,7 +2296,7 @@ class SqlParser(private val ion: IonSystem) : Parser {
}
}

validateTopLevelNodes(node, 0, 0)
validateTopLevelNodes(node = node, level = 0, topLevelTokenSeen = false)

return node.toExprNode()
}
Expand Down
2 changes: 2 additions & 0 deletions lang/test/org/partiql/lang/errors/ParserErrorsTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1740,6 +1740,8 @@ class ParserErrorsTest : TestBase() {
Property.TOKEN_TYPE to TokenType.KEYWORD,
Property.TOKEN_VALUE to ion.newSymbol("exec")))

// TODO: The token in the error message here should be "exec" instead of "undrop".
// Check this issue for more details. https://github.com/partiql/partiql-lang-kotlin/issues/372
@Test
fun execAtUnexpectedLocationInExpression() = checkInputThrowingParserException(
"SELECT * FROM (EXEC undrop 'foo')",
Expand Down