Failed to parse query with a slash after a 0.90 upgrade

[sgruhier]

Hi

I upgraded from 0.20 to 0.90 and a simple query I do failed when there is a slash in the query.
It's easy to test:

• create a simple element like in the documentation:

curl -XPUT 'http://localhost:9200/twitter/tweet/1' -d '{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elastic Search"
}'

• Run a query with a slash in it

curl -X GET 'http://localhost:9200/twitter/_search?from=0&size=12&pretty' -d '{
   "query":{
      "query_string":{
         "query":"user:kimchy/banon"
      }
   }
}'

• Check output
  works fine with 0.20. With that example you have 0 results
  returns this error with 0.90

  "error" : "SearchPhaseExecutionException[Failed to execute phase [query], total failure; shardFailures {[tJ5MGSY_RnOHfeAN2O8gnQ][twitter][2]: SearchParseException[[twitter][2]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\n   \"query\":{\n      \"query_string\":{\n         \"query\":\"user:kimchy/banon\"\n      }\n   }\n}]]]; nested: QueryParsingException[[twitter] Failed to parse query [user:kimchy/banon]]; nested: ParseException[Cannot parse 'user:kimchy/banon': Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; nested: TokenMgrError[Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; }{[tJ5MGSY_RnOHfeAN2O8gnQ][twitter][0]: SearchParseException[[twitter][0]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\n   \"query\":{\n      \"query_string\":{\n         \"query\":\"user:kimchy/banon\"\n      }\n   }\n}]]]; nested: QueryParsingException[[twitter] Failed to parse query [user:kimchy/banon]]; nested: ParseException[Cannot parse 'user:kimchy/banon': Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; nested: TokenMgrError[Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; }{[tJ5MGSY_RnOHfeAN2O8gnQ][twitter][1]: SearchParseException[[twitter][1]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\n   \"query\":{\n      \"query_string\":{\n         \"query\":\"user:kimchy/banon\"\n      }\n   }\n}]]]; nested: QueryParsingException[[twitter] Failed to parse query [user:kimchy/banon]]; nested: ParseException[Cannot parse 'user:kimchy/banon': Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; nested: TokenMgrError[Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; }{[tJ5MGSY_RnOHfeAN2O8gnQ][twitter][3]: SearchParseException[[twitter][3]: from[-1],size[-1]: Parse Failure [Failed to parse source [{\n   \"query\":{\n      \"query_string\":{\n         \"query\":\"user:kimchy/banon\"\n      }\n   }\n}]]]; nested: QueryParsingException[[twitter] Failed to parse query [user:kimchy/banon]]; nested: ParseException[Cannot parse 'user:kimchy/banon': Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; nested: TokenMgrError[Lexical error at line 1, column 18.  Encountered: <EOF> after : \"/banon\"]; }]",

Any idea?
Thanks

[sgruhier]

I update the issue with a full example with more details on the errors.

[sgruhier]

Ah, If I escape the slash like this:

curl -X GET 'http://localhost:9200/twitter/_search?from=0&size=12&pretty' -d '{
   "query":{
      "query_string":{
         "query":"user:kimchy\\/banon"
      }
   }
}'

It works!
So not a bug but a good thing to know

[dadoonet]

So should we close it?

[sgruhier]

yes

[s1monw]

FYI - this is due to regular expression support in the query parser since lucene 4.0 the slash indicates a regexp

[sgruhier]

Thanks for the explanation!
makes sense !

On May 3, 2013, at 3:18 PM, Simon Willnauer notifications@github.com wrote:

FYI - this is due to regular expression support in the query parser since lucene 4.0 the slash indicates a regexp

—
Reply to this email directly or view it on GitHub.

[sgruhier]

BTW, how regexp performance is in lucene 4? Should I avoid them?

[kimchy]

regex performance are considerably better in lucene 4, though obviously still slower compared to simple term queries or preparing the terms to search through during analysis on index time

[s1monw]

as usual, it depends. As a rule of thumb if you have a reasonable constant prefix these kind of queries are pretty fast. You should measure how it performs on your index and especially what kind of regular expressions you wanna use. Bottomline is they are useable in production since 4.0 (0.90) compared to lucene 3.6