fix: Fix BedrockChatGenerator to be able to handle multiple tool calls from one response

[sjrl]

Related Issues

• fixes AmazonBedrockChatGenerator + ToolInvoker don't handle multiple tool calls at once properly #1677
• fixes fix: AmazonBedrockChatGenerator does not pass Tool Call streaming chunk to streaming_callback #1680
• fixes fix: Inconsistent behavior between streaming and non-streaming in AmazonBedrockChatGenerator #1681

Proposed Changes:

Fixing Multi Tool Calling
Repairs the expected formatting of the bedrock messages.

Basically if a ChatMessage has multiple Tool Calls in it then the corresponding Tool Result ChatMessage must also contain both tool results. This is currently not possible because the ToolInvoker in Haystack returns a separate ChatMessage per Tool Call no matter if the original Tool Calls came one from one or multiple ChatMessages.

As a workaround, we retroactively repair the bedrock formatted messages to regroup the Tool Results based on how the Tool Calls are grouped.

This could also be "fixed" in the ToolInvoker, but I'm not sure if that would have adverse effects on our other integrations. Update: Nothing to fix in the ToolInvoker, most providers expect separate messages per tool result. So it's better to modify the behavior only in this integration.

Fixing Tool Call Streaming
Changed the internal logic of the integration to take Events from Bedrock, convert them into Haystack StreamingChunks that follow the same format as OpenAIChatGenerator and then pass the streaming chunk to the streaming callback. Now we will be able to reuse the print_streaming_chunk in Haystack main after a slight change is made. Here is an example of the output with the updated print_streaming_chunk coming in a separate PR.

Fixing Consistency Between Streaming and Non-Streaming
I updated the streaming processing to produce one ChatMessage like the non-streaming processing. This makes the most sense since this singular ChatMessage is converted into a single Bedrock formatted message.

How did you test it?

• Added unit tests
• Updated integration tests

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[sjrl]

@anakin87 I've assigned @vblagoje as the reviewer since he has more context.

However, @anakin87 I think you'd find the root of the problem to be potentially interesting which is described in the PR description. Basically this could have been also overcome by updating how the ToolInvoker works and by adding support for multiple tool results being stored in a ChatMessage dataclass.

[anakin87]

I'm not sure if I understand this correctly.

I think the choices we made in the past stem from the fact that most model providers (OpenAI, Google, Ollama) only accept messages from tool containing a single tool call result.
In this perspective, I think that handling this aspect specifically in the affected integrations is a viable option.

Please add more details if you think I have missed the point.

[sjrl]

I'm not sure if I understand this correctly.

I think the choices we made in the past stem from the fact that most model providers (OpenAI, Google, Ollama) only accept messages from tool containing a single tool call result. In this perspective, I think that handling this aspect specifically in the affected integrations is a viable option.

Please add more details if you think I have missed the point.

No I don't think you have. I ultimately agree that fixing it here makes the most sense. I just wasn't sure about this point

I think the choices we made in the past stem from the fact that most model providers (OpenAI, Google, Ollama) only accept messages from tool containing a single tool call result.

so thanks for the clarification!

In that case I agree leaving ToolInvoker as is makes the most sense.

[Amnah199]

@sjrl As discussed in the other PR, if this PR also addresses #1680 and #1681, please link them in the PR description for clarity.

[vblagoje]

I've checked with agent and it works in both streaming and non-streaming method - including multiple tool calls. ✅
The repair method is crucial and it deserves a bit more pydoc.
Suggested a small improvement for repair and other helper methods
I'll still take a look into streaming in detail, and verify tests.

[vblagoje]

I've completed the review. Finally we get this one 🔥
I'd use this opportunity to add some comments throughout the code so we can maintain it easier in the future. Great work 👍