[BUG] failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis] when value is in string

[Trisped]

Describe the bug

When sorting by a date field, if the field is not present on a record and "missing": "_first", "order": "asc" is specified (probably also applies to opposite) in the sort array, the returned sort array returns -9223372036854775808 for the sort value. If that value is passed back into the search_after array (for the next query) as a string, an error is returned: "failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]"

If it is passed as a number, it works fine.
If any other valid long value is passed as a string, it works fine.

Since Long.MIN_VALUE = -Long.MAX_VALUE - 1 and Long.MIN_VALUE = -9223372036854775808 I expect that this is an unhandled edge case.

Related component

Search:Resiliency

To Reproduce

1. Open the Dev Tools in the Dashboard
2. Run a query which sorts on a date field and put "-9223372036854775808" in the search_after field
3. Result is error
   Note: it may be required to have actual records which are missing the date field and you may need to have an ID field so the sort works correctly.

Sample query:

GET index-name/_search
{
  "size": 5,
  "query": {
    "bool": {
      "filter": [
        {
          "range": {
            "date": {
              "gte": "2024-11-26"
            }
          }
        },
        {
          "range": {
            "date": {
              "lte": "2024-12-28"
            }
          }
        }
      ]
    }
  },
  "_source": [
    "field1",
    "field2"
  ],
  "sort": [
    {
      "missingDatetime": {"missing": "_first", "order": "asc"}
    },
    {
      "id": "asc"
    }
  ],
  "search_after": [
    "-9223372036854775808",
    "cda11fb7-d250-41e1-945c-c31f204ac51c"
  ]
}

Expected behavior

Query returns next set of results (no exception).

Additional Details

Plugins
None

Screenshots

Host/Environment (please complete the following information):

• Version: OpenSearch 2.15

Additional context
Full exception:

{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]: [failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]]"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "can_match",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "<Index-Removed>",
        "node": "<Node Removed>",
        "reason": {
          "type": "parse_exception",
          "reason": "failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]: [failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]]",
          "caused_by": {
            "type": "illegal_argument_exception",
            "reason": "failed to parse date field [-9223372036854775808] with format [strict_date_optional_time||epoch_millis]",
            "caused_by": {
              "type": "date_time_parse_exception",
              "reason": "date_time_parse_exception: Failed to parse with all enclosed parsers"
            }
          }
        }
      }
    ]
  },
  "status": 400
}

[Trisped]

A search of the issue tracker for 9223372036854775808 turns up a few similar, though probably unrelated issues.

[sandeshkr419]

[Search Triage] @Trisped thanks for filing this - can you validate once if this is reproduce-able in recent 2.x release like 2.18?

[Trisped]

@sandeshkr419 Sorry, I don't have an environment where I can do that.
I did look through the release notes for 2.16, 2.17, and 2.18 and did not see anything related to this.

[msfroh]

Yeah --- this is probably an outstanding bug.

As a workaround, you might be able to follow the advice from #1490 and override the "missing" value to something higher but still negative (since the important thing isn't usually that it's -9223372036854775808, but rather that it's lower than any "real" values).

@Trisped, out of curiosity, does it end up working if you change "missing" to something higher, like -9223372036854775807? (Or even try doing a search_after with that slightly higher value?)

I fixed something that feels similar in #12676, but it was only addressing the sort side of things, not the search_after.

[Trisped]

@msfroh Yes, I tried searching with "-9223372036854775807" and it works.
As far as I can tell, it is just the one "-9223372036854775808" value.

[msfroh]

Awesome! Thanks for looking into it.

In a pinch, we could probably change the minimum missing value to -9223372036854775807 as a workaround. It's only off by a millisecond, after all. 😁

In truth, though, we should support -9223372036854775808 as an input. I'm pretty sure that this is a gap in the fix that I made for #12676.

Without reproducing it and getting a full stack trace, I'm guessing that it fails on this line. Specifically, it's peeled off the negative sign and then tries parsing the resulting positive number into a Long object, but it's Long.MAX_VALUE + 1, so it fails.

I added the following to the end of DateFormattersTests#testEpochMillisParser() and sure enough, it fails with exactly the exception you posted above:

        {
            Instant cannedInstant = Instant.ofEpochMilli(Long.MIN_VALUE);
            Instant instant = Instant.from(formatter.parse(Long.toString(Long.MIN_VALUE)));
            assertEquals(cannedInstant, instant);
        }