Skip to content

[rule based autotagging] Add Create Rule API Logic #17792

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: main
Choose a base branch
from

Conversation

ruai0511
Copy link
Contributor

@ruai0511 ruai0511 commented Apr 3, 2025

Description

This PR introduces the create Rule API Logic.

An exampe API request is:

curl -XPUT "localhost:9200/_rules/workload_group"  -H 'Content-Type: application/json' -d '     
{
    "description": "description",
     "index_pattern": ["log*", "event*"],
     "workload_group": "EITBzjFkQ6CA-semNWGtRQ"
}'

And the return would be

{
   "_id":"Kqqra5YBzZOW_5Q0ALDc",
   "description":"description",
   "index_pattern":["log*", "event*"],
   "workload_group":"EITBzjFkQ6CA-semNWGtRQ",
   "updated_at":"2025-04-25T06:38:10.787Z"
}

Rule Schema PR:
#17238
RFC:
#16797

Check List

  • Functionality includes testing.
  • API changes companion pull request created, if applicable.
  • Public documentation issue/PR created, if applicable.

Copy link
Contributor

github-actions bot commented Apr 3, 2025

❌ Gradle check result for 90e2b4c: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ruai0511 ruai0511 force-pushed the create-rule-api-PR branch from 90e2b4c to 85ea24e Compare April 4, 2025 00:27
Copy link
Contributor

github-actions bot commented Apr 4, 2025

❌ Gradle check result for 85ea24e: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ruai0511 ruai0511 force-pushed the create-rule-api-PR branch from 85ea24e to e215963 Compare April 4, 2025 01:27
Copy link
Contributor

github-actions bot commented Apr 4, 2025

❌ Gradle check result for e215963: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for e0c9c06: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

ruai0511 added 11 commits April 23, 2025 18:53
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
modify based on comments
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
Signed-off-by: Ruirui Zhang <[email protected]>
@ruai0511 ruai0511 force-pushed the create-rule-api-PR branch from e0c9c06 to 8f8ecd2 Compare April 25, 2025 08:05
Signed-off-by: Lingxi Chen <[email protected]>
@ruai0511 ruai0511 force-pushed the create-rule-api-PR branch from 8f8ecd2 to c6e7250 Compare April 25, 2025 08:07
Copy link
Contributor

❌ Gradle check result for c6e7250: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@ruai0511 ruai0511 force-pushed the create-rule-api-PR branch from c6e7250 to dbe2e18 Compare April 28, 2025 03:26
Copy link
Contributor

✅ Gradle check result for dbe2e18: SUCCESS

Copy link
Contributor

@kaushalmahi12 kaushalmahi12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments, LGTM! overall. We should focus on test coverage though

* {
* "description": "description1",
* "index_pattern": ["log*", "event*"],
* "query_group": "poOiU851RwyLYvV5lbvv5w"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets replace query_group with workload_group wherever applicable

@Override
public ActionRequestValidationException validate() {
try {
rule.getFeatureType().getFeatureValueValidator().validate(rule.getFeatureValue());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are chaining a lot of method calls in this statement, this can potentially result in NPE. Have we thought of the scenarios when this can be null ?

* "_id":"wi6VApYBoX5wstmtU_8l",
* "description":"description1",
* "index_pattern":["log*", "uvent*"],
* "query_group":"poOiU851RwyLYvV5lbvv5w",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NIT] name change

* Interface to check for rule duplication.
*/
@ExperimentalApi
public interface RuleDuplicateChecker {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think DuplicateRuleChecker is more appropriate name.

package org.opensearch.rule.autotagging;

/**
* Validates a feature value for a specific feature type
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets add more context for this class. I think this is not a simple string format and length checker but rather it validates the value against the pre-stored feature values (index, cluster state ....etc).

*/
public void createRule(CreateRuleRequest request, ActionListener<CreateRuleResponse> listener) {
try (ThreadContext.StoredContext ctx = getContext()) {
createIndexIfAbsent(new ActionListener<>() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can write this in a better way, Lot of action listener chaining is unnecessary

if (indexDoesNotExist) {
    createIndex(listener); //this is a method listener
} else {
    saveRule(listener);
}

private void createIndex(ActionListener<CreateRuleResponse> listener) {
     final CreateIndexRequest request = new CreateIndexRequest(indexName).settings(indexSettings);
            client.admin().indices().create(request, new ActionListener<>() {
                @Override
                public void onResponse(CreateIndexResponse response) {
                    if (!response.isAcknowledged()) {
                      throwError;
                    } else {
                        saveRule(listener);
                    }
                }

                @Override
                public void onFailure(Exception e) {
                    if (e instanceof ResourceAlreadyExistsException) {
                       saveRule(rule, listener);
                    } else {
                        logger.error("Failed to create index {}: {}", indexName, e.getMessage());
                        listener.onFailure(e);
                    }
                }
            });
}

Copy link
Contributor

❌ Gradle check result for 9a706c2: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

@kaushalmahi12 kaushalmahi12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! just need to address/answer the two questions

Comment on lines 55 to 58
FeatureValueValidator featureValueValidator = rule.getFeatureType().getFeatureValueValidator();
if (featureValueValidator != null) {
featureValueValidator.validate(rule.getFeatureValue());
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it will be abnormal behavior when feature validator is null hence we should at least log the instance or throw exception.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right. Yesterday, I thought it made sense for featureValueValidator to be null if a feature type didn’t need any validation. But thinking about it more now, it’s better if featureValueValidator is always present. If a feature type doesn’t actually need validation, its validate() method can just do nothing or return null (although i doubt if any feature value doesn't need validation)

}
@Override
public void onFailure(Exception e) {
if (e instanceof ResourceAlreadyExistsException) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we always get only this exception if the index already exists or e can be something different as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on this I believe it'll always be ResourceAlreadyExistsException

Signed-off-by: Ruirui Zhang <[email protected]>
@ruai0511 ruai0511 requested a review from a team as a code owner May 1, 2025 04:29
Copy link
Contributor

github-actions bot commented May 1, 2025

❌ Gradle check result for d70aa32: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants