Skip to content

Commit d1861c4

Browse files
committed
added doc
1 parent 92ec57e commit d1861c4

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

examples/server/README.md

+3
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,7 @@ The project is under active development, and we are [looking for feedback and co
152152
| `--api-key-file FNAME` | path to file containing API keys (default: none) |
153153
| `--ssl-key-file FNAME` | path to file a PEM-encoded SSL private key<br/>(env: LLAMA_ARG_SSL_KEY_FILE) |
154154
| `--ssl-cert-file FNAME` | path to file a PEM-encoded SSL certificate<br/>(env: LLAMA_ARG_SSL_CERT_FILE) |
155+
| `--chat-template-kwargs STRING` | JSON object containing additional params for the json template parser. Example: --chat_template_kwargs "{\"enable_thinking\":false}"<br/>(env: CHAT_TEMPLATE_KWARGS) |
155156
| `-to, --timeout N` | server read/write timeout in seconds (default: 600)<br/>(env: LLAMA_ARG_TIMEOUT) |
156157
| `--threads-http N` | number of threads used to process HTTP requests (default: -1)<br/>(env: LLAMA_ARG_THREADS_HTTP) |
157158
| `--cache-reuse N` | min chunk size to attempt reusing from the cache via KV shifting (default: 0)<br/>(env: LLAMA_ARG_CACHE_REUSE) |
@@ -1075,6 +1076,8 @@ See [OpenAI Chat Completions API documentation](https://platform.openai.com/docs
10751076

10761077
The `response_format` parameter supports both plain JSON output (e.g. `{"type": "json_object"}`) and schema-constrained JSON (e.g. `{"type": "json_object", "schema": {"type": "string", "minLength": 10, "maxLength": 100}}` or `{"type": "json_schema", "schema": {"properties": { "name": { "title": "Name", "type": "string" }, "date": { "title": "Date", "type": "string" }, "participants": { "items": {"type: "string" }, "title": "Participants", "type": "string" } } } }`), similar to other OpenAI-inspired API providers.
10771078

1079+
`chat_template_kwargs`: Allows sending additional parameters to the json templating system. For example: `{"enable_thinking": false}`
1080+
10781081
*Examples:*
10791082

10801083
You can use either Python `openai` library with appropriate checkpoints:

0 commit comments

Comments
 (0)