OpenAI: When using streaming with responses API, tool call arguments are parsed incorrectly.

[nidhi-nair]

Checked other resources

• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

I haven't generated a sample code, but here's a sample body that gets sent through

{
  "model": "gpt-4.1-mini",
  "temperature": 0.7,
  "top_p": 0.7,
  "stream": true,
  "tools": [
    {
      "type": "function",
      "name": "getRandomNumber",
      "parameters": {
        "type": "object",
        "properties": {
          "a": {
            "type": "string",
            "description": "The lower numeric bound for the random number"
          },
          "b": {
            "type": "string",
            "description": "The upper numeric bound for the random number"
          }
        },
        "required": [
          "a",
          "b"
        ]
      },
      "description": "This function can be used to find a random number within numeric bounds. (JS function) This function can be used to find a random number within bounds that are numeric"
    }
  ],
  "text": {},
  "input": [
    {
      "type": "message",
      "role": "user",
      "content": "{\"query\":\"Find a random number between 0 and 100\"}"
    }
  ]
}

Error Message and Stack Trace (if applicable)

Sample streaming response from OpenAI

event: response.output_item.added
data: {"type":"response.output_item.added","output_index":0,"item":{"id":"fc_id","type":"function_call","status":"in_progress","arguments":"","call_id":"call_foo","name":"getRandomNumber"}}

event: response.function_call_arguments.delta
data: {"type":"response.function_call_arguments.delta","item_id":"fc_id","output_index":0,"delta":"{\""}

event: response.function_call_arguments.delta
data: {"type":"response.function_call_arguments.delta","item_id":"fc_id","output_index":0,"delta":"a"}

...

event: response.output_item.done
data: {"type":"response.output_item.done","output_index":0,"item":{"id":"fc_id","type":"function_call","status":"completed","arguments":"{\"a\":\"0\",\"b\":\"100\"}","call_id":"call_foo","name":"getRandomNumber"}}

Description

The streaming response from Open AI for when a tool call is expected contains the full argument body in the event called response.output_item.done but we are currently expecting to find this information in response.output_item.added. As a result, the arguments are never parsed correctly in the response.

The following patch will fix the immediate issue, but I am unsure of side effects to this.

@@ -364,7 +364,7 @@ function _convertOpenAIResponsesDeltaToBaseMessageChunk(chunk) {
         chunk.item.type === "message") {
         id = chunk.item.id;
     }
-    else if (chunk.type === "response.output_item.added" &&
+    else if (chunk.type === "response.output_item.done" &&
         chunk.item.type === "function_call") {
         tool_call_chunks.push({
             type: "tool_call_chunk",

System Info

@langchain/openai@patch:@langchain/openai@npm%3A0.5.6#./.yarn/patches/@langchain-openai-npm-0.5.6-0735c9f38b.patch::version=0.5.6&hash=60d1c9&locator=appsmith%40workspace%3A.
   ├─ Instances: 1
   ├─ Version: 0.5.6

[benjamincburns]

Thanks for reporting this @nidhi-nair! I'll get a release with the fix out ASAP.

[benjamincburns]

@nidhi-nair - are you observing issues with tool calling when using the responses API, or is this something that you found by reading the code? FWIW, it's a bit split up in the code, but we do handle both the response.output_item.added and response.function_call_arguments.delta events.

We do it this way, rather than just capturing the final output on done, so that we can stream output chunks immediately as they become available.

In the meantime I'll verify that our integration tests exercise function calling with the responses API, and if that reveals any issues I'll get a fix out ASAP.

[benjamincburns]

From manual testing I do see an issue, but it's different from the one that you originally described. I'll fix it, but it'd be good to know if this matches what you're observing on your end.

The issue that I'm seeing occurs only when streaming. What happens is I get a tool call from the model, and handle it, but when I call stream with the ToolMessage in the message context I see an error like the following:

Error: 400 No tool output found for function call call_ZGwojVx5WgR6CZOZcAOmXs9u

This appears to be happening because in the streaming output we emit the responses item ID for the tool call in the call_id field rather than the call_id from the item itself. If I swap in the correct ID, things work as expected.

Fortunately this is a very simple one line fix, and I'll release it in a few minutes. That said - please please let me know if this doesn't replicate the behavior that you're seeing. If that's the case I'll gladly reopen and investigate further.

Either way, thanks once again for reporting!

[benjamincburns]

The fix for this went out in @langchain/openai v0.5.10

[nidhi-nair]

@benjamincburns this was a few weeks ago so I'm not entirely sure I'm remembering this correctly, but my observation was that the delta event processing was not aggregating the JSON in a way that could be parsed correctly. IIRC there was code somewhere that collected the args from the delta chunks and pushed it out if it made sense as a JSON, but for whatever reason the deltas never produced a proper aggregated JSON. Hence this workaround solution of adding the done event instead so we'd be guaranteed to find the args. Let me see if I can print out the args as was seen by that piece of code before it attempts to parse JSON.

This was from observation on behaviour as I was using the library, and I went digging into code afterward. 🙂 I'll check the updated lib for this other error, may be the aggregation logic was somehow linked?