use llguidance library for constraints (including json schemas)

README.md

@@ -89,7 +89,7 @@ Mistral.rs supports several model categories:
 **Easy**:
 - Lightweight OpenAI API compatible HTTP server
 - Python API
-- Grammar support with Regex and Yacc
+- Grammar support with JSON Schema, Regex, Lark, and Guidance via [LLGuidance library](https://github.com/microsoft/llguidance)
 - [ISQ](docs/ISQ.md) (In situ quantization): run `.safetensors` models directly from 🤗 Hugging Face by quantizing in-place
     - Enhance performance with an [imatrix](docs/IMATRIX.md)!
 

docs/HTTP.md

@@ -9,7 +9,7 @@ The API consists of the following endpoints. They can be viewed in your browser
 To support additional features, we have extended the completion and chat completion request objects. Both have the same keys added:
 
 - `top_k`: `int` | `null`. If non null, it is only relevant if positive.
-- `grammar`: `{"type" : "regex" | "yacc", "value": string}` or `null`. Grammar to use.
+- `grammar`: `{"type" : "regex" | "lark" | "json_schema" | "llguidance", "value": string}` or `null`. Grammar to use.
 - `adapters`: `array of string` | `null`. Adapter names to activate for this request.
 - `min_p`: `float` | `null`. If non null, it is only relevant if 1 >= min_p >= 0.
 

examples/python/json_schema.py

@@ -0,0 +1,34 @@
+from mistralrs import Runner, Which, ChatCompletionRequest, Architecture
+from json import dumps
+
+runner = Runner(
+    which=Which.Plain(
+        model_id="microsoft/Phi-3.5-mini-instruct",
+    ),
+    num_device_layers=["500"],
+)
+
+res = runner.send_chat_completion_request(
+    ChatCompletionRequest(
+        model="phi",
+        messages=[{"role": "user", "content": "Give me a sample address."}],
+        max_tokens=256,
+        temperature=0.1,
+        grammar_type="json_schema",
+        grammar=dumps(
+            {
+                "type": "object",
+                "properties": {
+                    "street": {"type": "string"},
+                    "city": {"type": "string"},
+                    "state": {"type": "string", "pattern": "^[A-Z]{2}$"},
+                    "zip": {"type": "integer", "minimum": 10000, "maximum": 99999},
+                },
+                "required": ["street", "city", "state", "zip"],
+                "additionalProperties": False,
+            }
+        ),
+    )
+)
+print(res.choices[0].message.content)
+print(res.usage)