Skip to content

[GRPC] Add terms query support in Search GRPC endpoint #17888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 11, 2025

Conversation

karenyrx
Copy link
Contributor

@karenyrx karenyrx commented Apr 10, 2025

This PR is stacked on top of #17895

Description

Implement Terms Query within GRPC Search API

Test Plan

  1. Unit tests
  2. Manually checking parity between the GRPC and HTTP APIs, e.g.
    a) TermsLookupField
grpcurl -import-path ~/opensearch -d @ -proto ~/opensearch/protos/services/search_service.proto -plaintext localhost:9400 org.opensearch.protobufs.services.SearchService/Search <<EOM
{
 "request_body": {
   "query": {
     "terms": {
       "boost": 1.0,
       "name": "fake-name",
       "value_type": "1",
       "terms_lookup_field_string_array_map": {
         "line_id": {
           "terms_lookup_field": {
             "index": "fake-index",
             "id": "fake-id",
             "path": "fake-path",
             "routing": "1",
             "store": true
           }
         }
       }
     }
   }
 }
}
curl -XGET "http://localhost:9200/my_index/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "terms": {
      "line_id": {
        "index": "fake-index",
        "id": "fake-id",
        "path": "fake-path",
        "routing": "1",
        "store": true
      },
      "_name": "fake-name",
      "boost": 1.0,
      "value_type": "bitmap"
    }
  }
}
' | jq

POJOs both the same:

SearchRequest{searchType=QUERY_THEN_FETCH, indices=[], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, expand_wildcards_hidden=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=null, allowPartialSearchResults=null, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"query":{"terms":{"line_id":{"index":"fake-index","id":"fake-id","path":"fake-path","routing":"1","store":true},"boost":1.0,"_name":"fake-name","value_type":"bitmap"}}}, cancelAfterTimeInterval=null, pipeline=null, phaseTook=null}

b) StringArray

grpcurl -import-path ~/opensearch -d @ -proto ~/opensearch/protos/services/search_service.proto -plaintext localhost:9400 org.opensearch.protobufs.services.SearchService/Search <<EOM

{
 "request_body": {
   "query": {
     "terms": {
       "terms_lookup_field_string_array_map": {
         "author": {
           "string_array": {
             "string_array": [
               "Author",
               "KX"
             ]
           }
         }
       }
     }
   }
 }
}
EOM
curl -XGET "http://localhost:9200/_search" -H 'Content-Type: application/json' -d'
{
  "query": {
    "terms": {
      "author": ["Author", "KX"]
    }
  }
}
' | jq

SearchRequest POJOs are both the same:

SearchRequest{searchType=QUERY_THEN_FETCH, indices=[], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, expand_wildcards_hidden=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=null, allowPartialSearchResults=null, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"query":{"terms":{"author":["Author","KX"],"boost":1.0}}}, cancelAfterTimeInterval=null, pipeline=null, phaseTook=null}

Related Issues

Resolves #[Issue number to be closed when this PR is merged]
#16783
#16784

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link
Contributor

❌ Gradle check result for a3f16c3: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@karenyrx karenyrx changed the title [GRPC] Add terms query and document proto refactoring [GRPC] Add terms query support in Search GRPC endpoint and bump org.opensearch:protobufs from 0.2.0 to 0.3.0 Apr 10, 2025
@karenyrx karenyrx force-pushed the just-terms branch 2 times, most recently from a6d53e5 to 8584024 Compare April 10, 2025 22:51
Copy link
Contributor

❕ Gradle check result for fb76ef6: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Copy link

codecov bot commented Apr 11, 2025

Codecov Report

Attention: Patch coverage is 85.00000% with 9 lines in your changes missing coverage. Please review.

Project coverage is 72.52%. Comparing base (a4bc4bf) to head (35f4b3b).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
...uest/search/query/TermsQueryBuilderProtoUtils.java 83.33% 3 Missing and 5 partials ⚠️
.../org/opensearch/index/query/TermsQueryBuilder.java 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main   #17888      +/-   ##
============================================
+ Coverage     72.44%   72.52%   +0.08%     
- Complexity    66801    66871      +70     
============================================
  Files          5455     5457       +2     
  Lines        309183   309243      +60     
  Branches      44987    45004      +17     
============================================
+ Hits         223981   224279     +298     
+ Misses        66873    66614     -259     
- Partials      18329    18350      +21     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@karenyrx karenyrx changed the title [GRPC] Add terms query support in Search GRPC endpoint and bump org.opensearch:protobufs from 0.2.0 to 0.3.0 [GRPC] Add terms query support in Search GRPC endpoint Apr 11, 2025
Signed-off-by: Karen Xu <[email protected]>
Signed-off-by: Karen Xu <[email protected]>
Copy link
Contributor

❌ Gradle check result for e89a57a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for e89a57a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for e89a57a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for e89a57a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for 35f4b3b: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❕ Gradle check result for 35f4b3b: UNSTABLE

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants