Skip to content

[Bug fixes] Fix ListIndexTool and SearchIndexTool #3720

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 12 commits into from
Apr 11, 2025

Conversation

pyek-bot
Copy link
Contributor

@pyek-bot pyek-bot commented Apr 9, 2025

Description

Fixes the description and mapping associated with these tools:

  • ListIndexTool
  • SearchIndexTool
  • IndexMappingTool

Check List

  • New functionality includes testing.
  • New functionality has been documented.
  • API changes companion pull request created.
  • Commits are signed per the DCO using --signoff.
  • Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

pyek-bot added 2 commits April 9, 2025 11:28
Signed-off-by: Pavan Yekbote <[email protected]>
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 18:31 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 18:31 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 18:31 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 18:31 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 21:14 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 21:14 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 21:14 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 21:14 — with GitHub Actions Error
@@ -693,6 +693,7 @@ private static void runTool(
tools.get(action).run(parameters, toolListener); // run tool
}
} catch (Exception e) {
log.error("Failed to run tool {}", action, e);
Copy link
Collaborator

@mingshl mingshl Apr 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the next line, the error message will be return to the listener, and it's going to be the same e.getMessage(). is the log needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

listener behavior might change over time and may not expose these details, so i think its better to explicitly log tool failure here irrespective of listener

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log is for debugging if later we need to deep dive to identify the root cause. And listener is for customer to be notified at that very moment.

final String[] indices = indexList.toArray(Strings.EMPTY_ARRAY);
if (indexList.isEmpty()) {
@SuppressWarnings("unchecked")
T empty = (T) ("There were no results searching the index parameter [" + parameters.get("index") + "].");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how do you know this guarantee that T is actually String. Otherwise this will cause a ClassCastException at runtime.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is existing code. But I think we can enhance the logic as we already know the indexList is empty

@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 9, 2025 22:23 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 19:43 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 19:43 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:00 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:00 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:00 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:00 — with GitHub Actions Failure
@@ -88,7 +88,7 @@ public MLToolSpec(StreamInput input) throws IOException {
}
this.tenantId = streamInputVersion.onOrAfter(VERSION_2_19_0) ? input.readOptionalString() : null;
if (input.available() > 0 && input.readBoolean()) {
parameters = input.readMap(StreamInput::readString, StreamInput::readOptionalString);
attributes = input.readMap(StreamInput::readString, StreamInput::readOptionalString);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it the attributes and parameters are reading the same map? what is the intention?

    if (input.readBoolean()) {
        parameters = input.readMap(StreamInput::readString, StreamInput::readOptionalString);
    }
    includeOutputInAgentResponse = input.readBoolean();
    if (input.getVersion().onOrAfter(MINIMAL_SUPPORTED_VERSION_FOR_TOOL_CONFIG) && input.readBoolean()) {
        configMap = input.readMap(StreamInput::readString, StreamInput::readOptionalString);
    }
    this.tenantId = streamInputVersion.onOrAfter(VERSION_2_19_0) ? input.readOptionalString() : null;
    if (input.available() > 0 && input.readBoolean()) {
        attributes = input.readMap(StreamInput::readString, StreamInput::readOptionalString);
    }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

earlier, parameters was being overwritten by attributes map as we were reassigning parameters (bug)

now, we are assigning it correctly to attributes

attributes map is the last item in the bytestream, readMap() is a method that allows populating a map from the bytestream

Copy link
Contributor Author

@pyek-bot pyek-bot Apr 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am noticing some weird behavior in a multi-node cluster, will validate and get back

***resolved, the issue was with reading attributes twice, this code is still valid

@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:44 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:44 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:44 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 10, 2025 20:44 — with GitHub Actions Failure
@pyek-bot
Copy link
Contributor Author

PS: 46481bd introduces a new agent type and does not have any test coverage. We plan to release this as an experimental feature. For GA, we will be adding UTs and ITs to cover most scenarios.

This PR fixes a few bugs from the original commit.

mingshl
mingshl previously approved these changes Apr 10, 2025
Copy link
Collaborator

@mingshl mingshl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approve to unblock 3.0 beta release. will watch at least one CI pass. Please add IT and UT at 3.0 GA to validate the functionalities

Copy link
Member

@yuye-aws yuye-aws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pyek-bot Can you update the unit tests with attributes on these 3 tools?

@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 11, 2025 04:36 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 11, 2025 04:36 — with GitHub Actions Error
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 11, 2025 04:36 — with GitHub Actions Failure
@pyek-bot pyek-bot had a problem deploying to ml-commons-cicd-env-require-approval April 11, 2025 04:36 — with GitHub Actions Error
@pyek-bot
Copy link
Contributor Author

pyek-bot commented Apr 11, 2025

@pyek-bot Can you update the unit tests with attributes on these 3 tools?

@yuye-aws Added some basic UT to validate if attributes has input_schema and strict and test their values.
Attributes is primarily used to tell the model what the expected schema is. It replaces a ${parameters.attributes.input_schema} placeholder within a tool template.

This was created in the previous commit: 46481bd

As mentioned in my previous comment, we plan to release this as an experimental feature. For GA, we will be adding UTs and ITs to cover most scenarios including attributes.

@mingshl mingshl merged commit 211c517 into opensearch-project:main Apr 11, 2025
5 of 9 checks passed
akolarkunnu pushed a commit to akolarkunnu/ml-commons that referenced this pull request Jun 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants