We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 66a2a0d commit 010f4d2Copy full SHA for 010f4d2
README.md
@@ -23,11 +23,13 @@ For example:
23
wget https://huggingface.co/sanjay920/Llama-3-8b-function-calling-alpha-v1.gguf/resolve/main/Llama-3-8b-function-calling-alpha-v1.gguf
24
```
25
26
-4. start server:
+4. start openai compatible server:
27
28
./llama-server -ngl 35 -m Llama-3-8b-function-calling-alpha-v1.gguf --port 1234 --host 0.0.0.0 -c 16000 --chat-template llama3
29
30
31
+5. That's it! Make sure you turn `stream` off when making api calls to the server, as streaming feature is not supported yet.
32
+
33
### Recent API changes
34
35
- [2024 Apr 21] `llama_token_to_piece` can now optionally render special tokens https://github.com/ggerganov/llama.cpp/pull/6807
0 commit comments