Skip to content

Commit 010f4d2

Browse files
committed
update readme
1 parent 66a2a0d commit 010f4d2

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,13 @@ For example:
2323
wget https://huggingface.co/sanjay920/Llama-3-8b-function-calling-alpha-v1.gguf/resolve/main/Llama-3-8b-function-calling-alpha-v1.gguf
2424
```
2525

26-
4. start server:
26+
4. start openai compatible server:
2727
```
2828
./llama-server -ngl 35 -m Llama-3-8b-function-calling-alpha-v1.gguf --port 1234 --host 0.0.0.0 -c 16000 --chat-template llama3
2929
```
3030

31+
5. That's it! Make sure you turn `stream` off when making api calls to the server, as streaming feature is not supported yet.
32+
3133
### Recent API changes
3234

3335
- [2024 Apr 21] `llama_token_to_piece` can now optionally render special tokens https://github.com/ggerganov/llama.cpp/pull/6807

0 commit comments

Comments
 (0)