Skip to content

[BUG] Mapping does not present error when data is incorrect #1110

Open
@MitchellGale

Description

@MitchellGale

What is the bug?
Invalid data format for mapping does not cause error in integration tests. It does cause an error when loading with curl manually.

Error when loading through curl.
 Bits-MacBook-Pro:resources mitchellg$ curl -H 'Content-Type: application/x-ndjson' -XPOST 'http://localhost:9200/text_keyword/_bulk?pretty' -u admin:admin --insecure --data-binary @text_keyword_index.json
{
  "took" : 231,
  "errors" : true,
  "items" : [
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "1",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 0,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "2",
        "status" : 400,
        "error" : {
          "type" : "mapper_parsing_exception",
          "reason" : "failed to parse",
          "caused_by" : {
            "type" : "json_parse_exception",
            "reason" : "Duplicate field 'typeTextFieldData'\n at [Source: (byte[])\"{\"typeKeyword\": \"key01\", \"typeText\": \"text01\", \"typeKeywordFieldNoFieldData\": \"keyword01\", \"typeTextFieldData\": \"keyFD01\", \"typeTextFieldData\":  \"textFD01OverTen\", \"textDataFieldNoFields\": \"textFDNF01\", \"int0\": 1}\"; line: 1, column: 143]"
          }
        }
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "3",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 1,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "4",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 2,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "5",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 3,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "6",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 4,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "7",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 5,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "text_keyword",
        "_id" : "8",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 6,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

How can one reproduce the bug?
Steps to reproduce the behavior:
Create mapping and load data that is incorrectly formatted.

Load
Data](https://github.com/opensearch-project/sql/blob/081882ce140666f577f1e6e7359fa2675a5269df/integ-test/src/test/resources/datatypes.json)
Mappings

in IT tests and observe there is no error from the invalid data..

What is the expected behavior?
IT tests should produce error instead of ignoring data.

What is your host/environment?

  • OS: MacOS
  • Version 12.6.1

Do you have any additional context?
This file, datatypes.json, was incorrectly formatted with brackets. Issue was not caught until PR 867.

Old data issue

Fix

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinginfrastructureChanges to infrastructure, testing, CI/CD, pipelines, etc.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions