Merge pull request #3822 from vespa-engine/bratseth/linguistics-query-type

Add 'linguistics' query.type

Author: bratseth

en/reference/query-api-reference.html

@@ -518,7 +518,7 @@ <h2 id="query-model">Query Model Parameters</h2>
         Selects the query language syntax of the
         <a href="#model.querystring">model.queryString</a> parameter:
         <code>all</code>, <code>weakAnd</code>, <code>any</code>, <code>phrase</code>,
-        <code>tokenize</code>, <code>web</code>, <code>yql</code> -
+        <code>tokenize</code>, <code>web</code>, <code>linguistics</code>, </code><code>yql</code> -
         refer to <a href="simple-query-language-reference.html">simple query language reference</a>.
         Also see <a href="query-language-reference.html#userinput">YQL grammar</a>.
       </p>

en/reference/simple-query-language-reference.html

@@ -42,6 +42,15 @@
           languages is not supported</li>
     </ul></td>
   </tr>
+  <tr>
+    <th>linguistics</th>
+    <td>Pass the entire string as-is to the linguistics tokenizer and collect all tokens returned into a
+      weakAnd item. This is suitable for applications that want full control over the linguistics processing,
+      typically through a configured linguistics component.
+      When this is used all aspects of linguistics processing including stemming and normalization is controlled
+      by the linguistics component, and such settings on schema fields are disregarded.
+      </td>
+  </tr>
   <tr>
     <th>phrase</th>
     <td>The words of the query is considered a phrase,