Skip to content

Merge main to 2.x branch #771

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 21 commits into from
Aug 17, 2022
Merged

Merge main to 2.x branch #771

merged 21 commits into from
Aug 17, 2022

Conversation

joshuali925
Copy link
Member

Description

[Describe what this change achieves]

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

MitchellGale and others added 21 commits August 2, 2022 14:23
@MitchellGale
Two single or double quote escapes single or double quote when string…
68663d1 
… is surrounded by same type of quote (opensearch-project#696)

Signed-off-by: mitchellg <[email protected]>

Co-authored-by: Andrew Carbonetto <[email protected]>
@Yury-Fridlyand
Remove BWC tests (opensearch-project#721)
d26cde3 
Signed-off-by: Yury Fridlyand <[email protected]>
@MitchellGale
Github Actions fix for reference to OpenSearch-Dashboard not existing…
6a99b56 
… outside of main OS sql project (opensearch-project#704)

Signed-off-by: mitchellg <[email protected]>
@forestmvey
Add Highlight In SQL (opensearch-project#717)
8f1b31d 
* Add support for highlight to parser and AstExpressionBuilder

Signed-off-by: MaxKsyunz <[email protected]>

* Add unit test for highlight in AstExpressionBuilder

Signed-off-by: MaxKsyunz <[email protected]>

* Add unit test for highlight in AstBuilderTest

Signed-off-by: MaxKsyunz <[email protected]>

* Support highlight as an Unresolved expression.

Signed-off-by: MaxKsyunz <[email protected]>

* Represent highlight as UnresolvedExpression.

Signed-off-by: MaxKsyunz <[email protected]>

* Support highlight in Analyzer.

Signed-off-by: MaxKsyunz <[email protected]>

* Treat highlight as a proper function in AST

In particular, highlightField is an expression now.

Signed-off-by: MaxKsyunz <[email protected]>

* Add support for highlight in Analyzer

HighlightFunction is converted to LogicalHighlight logical plan.

Signed-off-by: MaxKsyunz <[email protected]>

* Add a simple IT test for highlight.

Signed-off-by: MaxKsyunz <[email protected]>

* Register highlight function in the BuiltInFunctionRepository

Signed-off-by: MaxKsyunz <[email protected]>

* Partial support for highlight in physical plan.

Signed-off-by: MaxKsyunz <[email protected]>

* Add HighlightOperator.

Signed-off-by: MaxKsyunz <[email protected]>

* Highlight alpha complete.

Signed-off-by: MaxKsyunz <[email protected]>

* Initial implementation to upporting highlight('*')

Signed-off-by: forestmvey <[email protected]>

* Add support for multiple highlight calls in select statement.

Signed-off-by: forestmvey <[email protected]>

* Removed OpenSearchLogicalIndexScan highlightFields and dependencies. Improved test coverage and fixing checkstyle errors.

Signed-off-by: forestmvey <[email protected]>

* Added HighlightExpressionTest

Signed-off-by: forestmvey <[email protected]>

* Added javadocs, minor PR revisions, and fixed jacoco errors by improving coverage for OpenSearchIndexScan

Signed-off-by: forestmvey <[email protected]>

* Code cleanup, adding parsing failure tests, and adding tests for highlight acceptance as a string literal as well as qualified name.

Signed-off-by: forestmvey <[email protected]>

* Removing HighlightOperator functionality and unnecessary visitHighlight call in PhysicalPlanNodeVisitor..

Signed-off-by: forestmvey <[email protected]>

* Adding highlight function to functions.rst and removing unecessary function call in OpenSearchIndexScan.

Signed-off-by: forestmvey <[email protected]>

* Change highlight fields returned format to array list. Changed highlight all and wildcard to unsupported to open up output formatting changes for multiple returned highlight fields. Change tests for updated coverage and disable highlight all and wildcard tests.

Signed-off-by: forestmvey <[email protected]>

* Fix bug where invalid schema name was being used for returned highlight fields

Signed-off-by: forestmvey <[email protected]>

* Fix failing integration tests due to schema changes for highlight expression type.

Signed-off-by: forestmvey <[email protected]>

Co-authored-by: MaxKsyunz <[email protected]>
@penghuo
Bump version to 2.2.0 (opensearch-project#729)
df32384 
Signed-off-by: penghuo <[email protected]>
@joshuali925
Add release notes for 2.2.0.0 (opensearch-project#730)
1df89e3 
Signed-off-by: Joshua Li <[email protected]>
@penghuo
Bump workbench version to 2.2.0.0 (opensearch-project#736)
792d3ff 
Signed-off-by: penghuo <[email protected]>
@MitchellGale @Yury-Fridlyand
Adding documentation about double quote implementation (opensearch-pr…
921a28c 
…oject#723)

Signed-off-by: mitchellg <[email protected]>
Co-authored-by: Yury Fridlyand <[email protected]>
@joshuali925
Update snapshots for jest tests on workbench (opensearch-project#740)
e6a5ac9 
Signed-off-by: Joshua Li <[email protected]>
@dai-chen
Bump JDBC driver's dependency version (opensearch-project#735)
29f1c52 
* Bump wiremock version

Signed-off-by: Chen Dai <[email protected]>

* Bump guava and json-smart version

Signed-off-by: Chen Dai <[email protected]>

* Bump httpclient version

Signed-off-by: Chen Dai <[email protected]>
@dai-chen
Bump Jackson dependency version (opensearch-project#745)
6fb3fac 
* Bump jackson version and add version config in root gradle script

Signed-off-by: Chen Dai <[email protected]>

* Bump jackson version in JDBC driver

Signed-off-by: Chen Dai <[email protected]>

Signed-off-by: Chen Dai <[email protected]>
@seankao-az
Extend query size limit using scroll (opensearch-project#716)
f3c9c29 
* add maxResultWindow to LogicalRelation

Signed-off-by: Sean Kao <[email protected]>

* add maxResultWindow to OpenSearchLogicalIndexScan

Signed-off-by: Sean Kao <[email protected]>

* OpenSearchRequestBuilder init

Signed-off-by: Sean Kao <[email protected]>

* request builder: push down and build

Signed-off-by: Sean Kao <[email protected]>

* plan.build() for building request

Signed-off-by: Sean Kao <[email protected]>

* maxResultWindow for test utils

Signed-off-by: Sean Kao <[email protected]>

* fix style

Signed-off-by: Sean Kao <[email protected]>

* remove plan.build()

Signed-off-by: Sean Kao <[email protected]>

* fetch result in batches

Signed-off-by: Sean Kao <[email protected]>

* get index.max_result_window settings

Signed-off-by: Sean Kao <[email protected]>

* use index.max_result_window to decide scroll

Signed-off-by: Sean Kao <[email protected]>

* maxResultWindow for aggregation

Signed-off-by: Sean Kao <[email protected]>

* fix fetch size & for aggregation query

Signed-off-by: Sean Kao <[email protected]>

* fix rest client get max result window

Signed-off-by: Sean Kao <[email protected]>

* remove maxResultWindow from logical plan

Signed-off-by: Sean Kao <[email protected]>

* get max result window when building physical plan

Signed-off-by: Sean Kao <[email protected]>

* move source builder init to request builder

Signed-off-by: Sean Kao <[email protected]>

* fix max result window for test & rest client

Signed-off-by: Sean Kao <[email protected]>

* include request builder in equal comparison

Signed-off-by: Sean Kao <[email protected]>

* rename getIndexMaxResultWindows

Signed-off-by: Sean Kao <[email protected]>

* open search rest client test

Signed-off-by: Sean Kao <[email protected]>

* test: request builder, scroll index scan

Signed-off-by: Sean Kao <[email protected]>

* fix style

Signed-off-by: Sean Kao <[email protected]>

* remove getMaxResultWindow from base Table

Signed-off-by: Sean Kao <[email protected]>

* remove unused import from OpenSearchIndexScan

Signed-off-by: Sean Kao <[email protected]>

* test index scan

Signed-off-by: Sean Kao <[email protected]>

* integ test for head command

Signed-off-by: Sean Kao <[email protected]>

* keep request query size for aggregation

Signed-off-by: Sean Kao <[email protected]>

* fix rest client test coverage

Signed-off-by: Sean Kao <[email protected]>

* test for node client

Signed-off-by: Sean Kao <[email protected]>

* test node client default settings

Signed-off-by: Sean Kao <[email protected]>

* change Elasticsearch to OpenSearch in comment

Signed-off-by: Sean Kao <[email protected]>

* fix comments

Signed-off-by: Sean Kao <[email protected]>

* more test for Head IT

Signed-off-by: Sean Kao <[email protected]>

* ignore some head IT

Signed-off-by: Sean Kao <[email protected]>

Signed-off-by: Sean Kao <[email protected]>
@Yury-Fridlyand
Replace 2x LogUtils by QueryContext (opensearch-project#747)
deececb 
Signed-off-by: Yury Fridlyand <[email protected]>
@MitchellGale
Add any case of arguments in relevancy based functions to be allowed (o…
ac9b63a 
…pensearch-project#744)

* Added new functionality around argument names not case specific for relevancy based functions. Added unit tests to cover updates.

Signed-off-by: MitchellGale-BitQuill <[email protected]>

* Fixed test cases that used uppercase values in arguments.

Signed-off-by: MitchellGale-BitQuill <[email protected]>

Signed-off-by: MitchellGale-BitQuill <[email protected]>
Callout distinction between original and current SQL engines. (opense…
45d84d0 
…arch-project#728)

Signed-off-by: MaxKsyunz <[email protected]>

Signed-off-by: MaxKsyunz <[email protected]>
@joshuali925
Update PPL docs link for workbench (opensearch-project#758)
95a24b2 
Signed-off-by: Joshua Li <[email protected]>

Signed-off-by: Joshua Li <[email protected]>
@Yury-Fridlyand
Add maketime and makedate (opensearch-project#102) (opensearch-pr…
9f602c3 
…oject#755)

Signed-off-by: Yury Fridlyand <[email protected]>

Signed-off-by: Yury Fridlyand <[email protected]>
@joshuali925
Upgrade yarn dependencies for workbench (opensearch-project#765)
23a4a88 
Signed-off-by: Joshua Li <[email protected]>
@penghuo
Schedule request in worker thread (opensearch-project#748)
ce15448 
Signed-off-by: penghuo <[email protected]>
@joshuali925
Merge branch 'main' into 2.x
8729c32
@joshuali925
Resolve conflicts
8185127 
Signed-off-by: Joshua Li <[email protected]>
@joshuali925 joshuali925 requested a review from a team as a code owner August 17, 2022 18:02
@codecov-commenter
Copy link

Codecov Report

Merging #771 (8185127) into 2.x (dabe1d7) will decrease coverage by 34.98%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##                2.x     #771       +/-   ##
=============================================
- Coverage     97.74%   62.76%   -34.99%     
=============================================
  Files           273       10      -263     
  Lines          7021      658     -6363     
  Branches        442      118      -324     
=============================================
- Hits           6863      413     -6450     
- Misses          157      192       +35     
- Partials          1       53       +52
Flag Coverage Δ
query-workbench 62.76% <0.00%> (?)
sql-engine ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
workbench/public/components/Main/main.tsx 53.00% <0.00%> (ø)
...ain/java/org/opensearch/sql/analysis/Analyzer.java
...rg/opensearch/sql/analysis/ExpressionAnalyzer.java
...ensearch/sql/expression/ExpressionNodeVisitor.java
...arch/sql/expression/datetime/DateTimeFunction.java
...h/sql/expression/function/BuiltinFunctionName.java
...h/sql/expression/function/OpenSearchFunctions.java
...opensearch/sql/planner/logical/LogicalPlanDSL.java
...ch/sql/planner/logical/LogicalPlanNodeVisitor.java
...ensearch/sql/planner/physical/PhysicalPlanDSL.java
... and 274 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@dai-chen
Copy link
Collaborator

@joshuali925 Thanks for syncing main and 2.x. I found some PRs are already merged to 2.x earlier or being reviewed in auto backport PR. Probably in future we need to make sure all PRs merged to main have backport PR and merged to ?.x automatically. For now, should we remove those and cherry-pick the remaining?

@penghuo
Copy link
Collaborator

penghuo commented Aug 17, 2022

@joshuali925 Thanks for syncing main and 2.x. I found some PRs are already merged to 2.x earlier or being reviewed in auto backport PR. Probably in future we need to make sure all PRs merged to main have backport PR and merged to ?.x automatically. For now, should we remove those and cherry-pick the remaining?

Yes, we should close all the auto-porting PR

@penghuo penghuo mentioned this pull request Aug 17, 2022
Closed
@joshuali925 joshuali925 merged commit 9819635 into opensearch-project:2.x Aug 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@penghuo penghuo penghuo approved these changes

@dai-chen dai-chen dai-chen approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@joshuali925 @codecov-commenter @dai-chen @penghuo @MitchellGale @Yury-Fridlyand @forestmvey @seankao-az