Skip to content

CohereChatGenerator using command-r model doesn't work with Agent #1693

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vblagoje opened this issue May 2, 2025 · 0 comments
Open

CohereChatGenerator using command-r model doesn't work with Agent #1693

vblagoje opened this issue May 2, 2025 · 0 comments
Labels
bug Something isn't working P2

Comments

@vblagoje
Copy link
Member

vblagoje commented May 2, 2025

Describe the bug
command-r (Cohere model) fails during tool calling in a Haystack Agent pipeline. Although our test suite passes when evaluating tool use with the live model, actual runs frequently trigger a PipelineRuntimeError due to the model not returning valid tool calls or responses.

Error observed:

haystack.core.errors.PipelineRuntimeError: The following component failed to run:
Component name: 'chat_generator'
Component type: 'CohereChatGenerator'
Error: status_code: 422, body: {'message': 'No valid tool call or response generated'}

To Reproduce
Run a Haystack Agent pipeline using CohereChatGenerator with the command-r model and a valid tool registered. The agent setup completes, but at runtime, the model fails to return a proper tool call.

This works with command-r-plus, and with command-a-03-2025 but fails consistently with command-r.

Describe your environment (please complete the following information):

  • OS: macOS 14.4.1
  • Haystack version: 2.13.0
  • Integration version: cohere-haystack-3.0.0

Notes:
At first I thought this must have been related to MCP as we have tool tests in test suite but I eliminated that possibility by testing an Agent instance with single Tool used in integration tests (fake weather tool) and that doesn't work either so it is likely something related to Agent, tools and command-r combination

@vblagoje vblagoje added the bug Something isn't working label May 2, 2025
@julian-risch julian-risch added the P2 label May 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working P2
Projects
None yet
Development

No branches or pull requests

2 participants