Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using remote Llama-cpp, endpoint is "completion" not "completions" #4991

Closed
3 tasks done
jlaura opened this issue Apr 4, 2025 · 1 comment · Fixed by #5030
Closed
3 tasks done

Using remote Llama-cpp, endpoint is "completion" not "completions" #4991

jlaura opened this issue Apr 4, 2025 · 1 comment · Fixed by #5030
Assignees
Labels
ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior

Comments

@jlaura
Copy link

jlaura commented Apr 4, 2025

Before submitting your bug report

Relevant environment info

Using remote Llama-cpp, endpoint is "completion" not "completions". The error is a 404. I tested with llama-serve and llama-cpp-python. Both work with http://localhost:23451/v1/completions and CURL.

Error: `HTTP 404 Not Found from http://127.0.0.1:23452/v1/completion {"detail":"Not Found"}`

Environment:
- OS: OS X
- Continue version: 1.0.5 (pre-release)
- IDE version: VSCode 1.98.2
- config:
  
models:
  - name: CodeLlama7bChat
    provider: llama.cpp
    model: codellama-7b-instruct-Q6_K
    apiBase: http://127.0.0.1:23452/v1
    roles:
      - chat
      - edit
      - apply
tabAutoCompleteModel:
    title: DeekSeekCoder
    provider: llama.cpp
    model: deepseek-coder:1.3b-base-q6_K
    apiBase: http://127.0.0.1:23451/v1
context:
  - provider: code
  - provider: docs
  - provider: diff
  - provider: terminal

  
I also tested with:


    env:
      useLegacyCompletionsEndpoint: false

in the config (based on an open PR).

Description

No response

To reproduce

No response

Log output

@dosubot dosubot bot added ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior labels Apr 4, 2025
@Patrick-Erichsen
Copy link
Collaborator

Thanks for the heads up here @jlaura ! I think this should fix it: #5030

If you're able to pull down and try it locally that would be appreciated.

I'm a little worried that this will break existing functionality for other Llama-cpp users however, given that code is ~15mo old and I don't think we've had issues with it previously.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ide:vscode Relates specifically to VS Code extension kind:bug Indicates an unexpected problem or unintended behavior
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants