Skip to content

Commit 0ebddab

Browse files
author
Erick Friis
authored
docs, core: error messaging [wip] (langchain-ai#27397)
1 parent 202d7f6 commit 0ebddab

File tree

12 files changed

+644
-6
lines changed

12 files changed

+644
-6
lines changed

‎docs/docs/troubleshooting/errors/INVALID_PROMPT_INPUT.mdx

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# INVALID_PROMPT_INPUT
2+
3+
A [prompt template](/docs/concepts#prompt-templates) received missing or invalid input variables.
4+
5+
## Troubleshooting
6+
7+
The following may help resolve this error:
8+
9+
- Double-check your prompt template to ensure that it is correct.
10+
- If you are using the default f-string format and you are using curly braces `{` anywhere in your template, they should be double escaped like this: `{{` (and if you want to render a double curly brace, you should use four curly braces: `{{{{`).
11+
- If you are using a [`MessagesPlaceholder`](/docs/concepts/#messagesplaceholder), make sure that you are passing in an array of messages or message-like objects.
12+
- If you are using shorthand tuples to declare your prompt template, make sure that the variable name is wrapped in curly braces (`["placeholder", "{messages}"]`).
13+
- Try viewing the inputs into your prompt template using [LangSmith](https://docs.smith.langchain.com/) or log statements to confirm they appear as expected.
14+
- If you are pulling a prompt from the [LangChain Prompt Hub](https://smith.langchain.com/prompts), try pulling and logging it or running it in isolation with a sample input to confirm that it is what you expect.

‎docs/docs/troubleshooting/errors/INVALID_TOOL_RESULTS.ipynb

Lines changed: 272 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/docs/troubleshooting/errors/MESSAGE_COERCION_FAILURE.ipynb

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# MESSAGE_COERCION_FAILURE\n",
8+
"\n",
9+
"Instead of always requiring instances of `BaseMessage`, several modules in LangChain take `MessageLikeRepresentation`, which is defined as:"
10+
]
11+
},
12+
{
13+
"cell_type": "code",
14+
"execution_count": 4,
15+
"metadata": {},
16+
"outputs": [],
17+
"source": [
18+
"from typing import Union\n",
19+
"\n",
20+
"from langchain_core.prompts.chat import (\n",
21+
" BaseChatPromptTemplate,\n",
22+
" BaseMessage,\n",
23+
" BaseMessagePromptTemplate,\n",
24+
")\n",
25+
"\n",
26+
"MessageLikeRepresentation = Union[\n",
27+
" Union[BaseMessagePromptTemplate, BaseMessage, BaseChatPromptTemplate],\n",
28+
" tuple[\n",
29+
" Union[str, type],\n",
30+
" Union[str, list[dict], list[object]],\n",
31+
" ],\n",
32+
" str,\n",
33+
"]"
34+
]
35+
},
36+
{
37+
"cell_type": "markdown",
38+
"metadata": {},
39+
"source": [
40+
"These include OpenAI style message objects (`{ role: \"user\", content: \"Hello world!\" }`),\n",
41+
"tuples, and plain strings (which are converted to [`HumanMessages`](/docs/concepts#humanmessage)).\n",
42+
"\n",
43+
"If one of these modules receives a value outside of one of these formats, you will receive an error like the following:"
44+
]
45+
},
46+
{
47+
"cell_type": "code",
48+
"execution_count": 5,
49+
"metadata": {},
50+
"outputs": [
51+
{
52+
"ename": "ValueError",
53+
"evalue": "Message dict must contain 'role' and 'content' keys, got {'role': 'HumanMessage', 'random_field': 'random value'}",
54+
"output_type": "error",
55+
"traceback": [
56+
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
57+
"\u001b[0;31mKeyError\u001b[0m Traceback (most recent call last)",
58+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:318\u001b[0m, in \u001b[0;36m_convert_to_message\u001b[0;34m(message)\u001b[0m\n\u001b[1;32m 317\u001b[0m \u001b[38;5;66;03m# None msg content is not allowed\u001b[39;00m\n\u001b[0;32m--> 318\u001b[0m msg_content \u001b[38;5;241m=\u001b[39m \u001b[43mmsg_kwargs\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpop\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mcontent\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m \u001b[38;5;129;01mor\u001b[39;00m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 319\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n",
59+
"\u001b[0;31mKeyError\u001b[0m: 'content'",
60+
"\nThe above exception was the direct cause of the following exception:\n",
61+
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
62+
"Cell \u001b[0;32mIn[5], line 10\u001b[0m\n\u001b[1;32m 3\u001b[0m uncoercible_message \u001b[38;5;241m=\u001b[39m {\n\u001b[1;32m 4\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mHumanMessage\u001b[39m\u001b[38;5;124m\"\u001b[39m,\n\u001b[1;32m 5\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrandom_field\u001b[39m\u001b[38;5;124m\"\u001b[39m: \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrandom value\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 6\u001b[0m }\n\u001b[1;32m 8\u001b[0m model \u001b[38;5;241m=\u001b[39m ChatAnthropic(model\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mclaude-3-5-sonnet-20240620\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m---> 10\u001b[0m \u001b[43mmodel\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43minvoke\u001b[49m\u001b[43m(\u001b[49m\u001b[43m[\u001b[49m\u001b[43muncoercible_message\u001b[49m\u001b[43m]\u001b[49m\u001b[43m)\u001b[49m\n",
63+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/language_models/chat_models.py:287\u001b[0m, in \u001b[0;36mBaseChatModel.invoke\u001b[0;34m(self, input, config, stop, **kwargs)\u001b[0m\n\u001b[1;32m 275\u001b[0m \u001b[38;5;28;01mdef\u001b[39;00m \u001b[38;5;21minvoke\u001b[39m(\n\u001b[1;32m 276\u001b[0m \u001b[38;5;28mself\u001b[39m,\n\u001b[1;32m 277\u001b[0m \u001b[38;5;28minput\u001b[39m: LanguageModelInput,\n\u001b[0;32m (...)\u001b[0m\n\u001b[1;32m 281\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs: Any,\n\u001b[1;32m 282\u001b[0m ) \u001b[38;5;241m-\u001b[39m\u001b[38;5;241m>\u001b[39m BaseMessage:\n\u001b[1;32m 283\u001b[0m config \u001b[38;5;241m=\u001b[39m ensure_config(config)\n\u001b[1;32m 284\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m cast(\n\u001b[1;32m 285\u001b[0m ChatGeneration,\n\u001b[1;32m 286\u001b[0m \u001b[38;5;28mself\u001b[39m\u001b[38;5;241m.\u001b[39mgenerate_prompt(\n\u001b[0;32m--> 287\u001b[0m [\u001b[38;5;28;43mself\u001b[39;49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43m_convert_input\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m)\u001b[49m],\n\u001b[1;32m 288\u001b[0m stop\u001b[38;5;241m=\u001b[39mstop,\n\u001b[1;32m 289\u001b[0m callbacks\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mcallbacks\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[1;32m 290\u001b[0m tags\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mtags\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[1;32m 291\u001b[0m metadata\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mmetadata\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[1;32m 292\u001b[0m run_name\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mget(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_name\u001b[39m\u001b[38;5;124m\"\u001b[39m),\n\u001b[1;32m 293\u001b[0m run_id\u001b[38;5;241m=\u001b[39mconfig\u001b[38;5;241m.\u001b[39mpop(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mrun_id\u001b[39m\u001b[38;5;124m\"\u001b[39m, \u001b[38;5;28;01mNone\u001b[39;00m),\n\u001b[1;32m 294\u001b[0m \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mkwargs,\n\u001b[1;32m 295\u001b[0m )\u001b[38;5;241m.\u001b[39mgenerations[\u001b[38;5;241m0\u001b[39m][\u001b[38;5;241m0\u001b[39m],\n\u001b[1;32m 296\u001b[0m )\u001b[38;5;241m.\u001b[39mmessage\n",
64+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/language_models/chat_models.py:267\u001b[0m, in \u001b[0;36mBaseChatModel._convert_input\u001b[0;34m(self, input)\u001b[0m\n\u001b[1;32m 265\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m StringPromptValue(text\u001b[38;5;241m=\u001b[39m\u001b[38;5;28minput\u001b[39m)\n\u001b[1;32m 266\u001b[0m \u001b[38;5;28;01melif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(\u001b[38;5;28minput\u001b[39m, Sequence):\n\u001b[0;32m--> 267\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m ChatPromptValue(messages\u001b[38;5;241m=\u001b[39m\u001b[43mconvert_to_messages\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43minput\u001b[39;49m\u001b[43m)\u001b[49m)\n\u001b[1;32m 268\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[1;32m 269\u001b[0m msg \u001b[38;5;241m=\u001b[39m (\n\u001b[1;32m 270\u001b[0m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mInvalid input type \u001b[39m\u001b[38;5;132;01m{\u001b[39;00m\u001b[38;5;28mtype\u001b[39m(\u001b[38;5;28minput\u001b[39m)\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m. \u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 271\u001b[0m \u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMust be a PromptValue, str, or list of BaseMessages.\u001b[39m\u001b[38;5;124m\"\u001b[39m\n\u001b[1;32m 272\u001b[0m )\n",
65+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:348\u001b[0m, in \u001b[0;36mconvert_to_messages\u001b[0;34m(messages)\u001b[0m\n\u001b[1;32m 346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(messages, PromptValue):\n\u001b[1;32m 347\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m messages\u001b[38;5;241m.\u001b[39mto_messages()\n\u001b[0;32m--> 348\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43m[\u001b[49m\u001b[43m_convert_to_message\u001b[49m\u001b[43m(\u001b[49m\u001b[43mm\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mm\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mmessages\u001b[49m\u001b[43m]\u001b[49m\n",
66+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:348\u001b[0m, in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m 346\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28misinstance\u001b[39m(messages, PromptValue):\n\u001b[1;32m 347\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m messages\u001b[38;5;241m.\u001b[39mto_messages()\n\u001b[0;32m--> 348\u001b[0m \u001b[38;5;28;01mreturn\u001b[39;00m [\u001b[43m_convert_to_message\u001b[49m\u001b[43m(\u001b[49m\u001b[43mm\u001b[49m\u001b[43m)\u001b[49m \u001b[38;5;28;01mfor\u001b[39;00m m \u001b[38;5;129;01min\u001b[39;00m messages]\n",
67+
"File \u001b[0;32m~/langchain/oss-py/libs/core/langchain_core/messages/utils.py:321\u001b[0m, in \u001b[0;36m_convert_to_message\u001b[0;34m(message)\u001b[0m\n\u001b[1;32m 319\u001b[0m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mKeyError\u001b[39;00m \u001b[38;5;28;01mas\u001b[39;00m e:\n\u001b[1;32m 320\u001b[0m msg \u001b[38;5;241m=\u001b[39m \u001b[38;5;124mf\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mMessage dict must contain \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mrole\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m and \u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mcontent\u001b[39m\u001b[38;5;124m'\u001b[39m\u001b[38;5;124m keys, got \u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmessage\u001b[38;5;132;01m}\u001b[39;00m\u001b[38;5;124m\"\u001b[39m\n\u001b[0;32m--> 321\u001b[0m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(msg) \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01me\u001b[39;00m\n\u001b[1;32m 322\u001b[0m _message \u001b[38;5;241m=\u001b[39m _create_message_from_message_type(\n\u001b[1;32m 323\u001b[0m msg_type, msg_content, \u001b[38;5;241m*\u001b[39m\u001b[38;5;241m*\u001b[39mmsg_kwargs\n\u001b[1;32m 324\u001b[0m )\n\u001b[1;32m 325\u001b[0m \u001b[38;5;28;01melse\u001b[39;00m:\n",
68+
"\u001b[0;31mValueError\u001b[0m: Message dict must contain 'role' and 'content' keys, got {'role': 'HumanMessage', 'random_field': 'random value'}"
69+
]
70+
}
71+
],
72+
"source": [
73+
"from langchain_anthropic import ChatAnthropic\n",
74+
"\n",
75+
"uncoercible_message = {\"role\": \"HumanMessage\", \"random_field\": \"random value\"}\n",
76+
"\n",
77+
"model = ChatAnthropic(model=\"claude-3-5-sonnet-20240620\")\n",
78+
"\n",
79+
"model.invoke([uncoercible_message])"
80+
]
81+
},
82+
{
83+
"cell_type": "markdown",
84+
"metadata": {},
85+
"source": [
86+
"## Troubleshooting\n",
87+
"\n",
88+
"The following may help resolve this error:\n",
89+
"\n",
90+
"- Ensure that all inputs to chat models are an array of LangChain message classes or a supported message-like.\n",
91+
" - Check that there is no stringification or other unexpected transformation occuring.\n",
92+
"- Check the error's stack trace and add log or debugger statements."
93+
]
94+
},
95+
{
96+
"cell_type": "markdown",
97+
"metadata": {},
98+
"source": []
99+
}
100+
],
101+
"metadata": {
102+
"kernelspec": {
103+
"display_name": ".venv",
104+
"language": "python",
105+
"name": "python3"
106+
},
107+
"language_info": {
108+
"codemirror_mode": {
109+
"name": "ipython",
110+
"version": 3
111+
},
112+
"file_extension": ".py",
113+
"mimetype": "text/x-python",
114+
"name": "python",
115+
"nbconvert_exporter": "python",
116+
"pygments_lexer": "ipython3",
117+
"version": "3.11.4"
118+
}
119+
},
120+
"nbformat": 4,
121+
"nbformat_minor": 2
122+
}

‎docs/docs/troubleshooting/errors/MODEL_AUTHENTICATION.mdx

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
# MODEL_AUTHENTICATION
2+
3+
Your model provider is denying you access to their service.
4+
5+
## Troubleshooting
6+
7+
The following may help resolve this error:
8+
9+
- Confirm that your API key or other credentials are correct.
10+
- If you are relying on an environment variable to authenticate, confirm that the variable name is correct and that it has a value set.
11+
- Note that environment variables can also be set by packages like `dotenv`.
12+
- For models, you can try explicitly passing an `api_key` parameter to rule out any environment variable issues like this:
13+
14+
```python
15+
model = ChatOpenAI(api_key="YOUR_KEY_HERE")
16+
```
17+
18+
- If you are using a proxy or other custom endpoint, make sure that your custom provider does not expect an alternative authentication scheme.

‎docs/docs/troubleshooting/errors/MODEL_NOT_FOUND.mdx

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# MODEL_NOT_FOUND
2+
3+
The model name you have specified is not acknowledged by your provider.
4+
5+
## Troubleshooting
6+
7+
The following may help resolve this error:
8+
9+
- Double check the model string you are passing in.
10+
- If you are using a proxy or other alternative host with a model wrapper, confirm that the permitted model names are not restricted or altered.

‎docs/docs/troubleshooting/errors/MODEL_RATE_LIMIT.mdx

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# MODEL_RATE_LIMIT
2+
3+
You have hit the maximum number of requests that a model provider allows over a given time period and are being temporarily blocked.
4+
Generally, this error is temporary and your limit will reset after a certain amount of time.
5+
6+
## Troubleshooting
7+
8+
The following may help resolve this error:
9+
10+
- Contact your model provider and ask for a rate limit increase.
11+
- If many of your incoming requests are the same, utilize [model response caching](/docs/how_to/chat_model_caching/).
12+
- Spread requests across different providers if your application allows it.
13+
- Use a [`rate_limiter`](/docs/how_to/chat_model_rate_limiting/) to control the rate of requests to the model.

0 commit comments

Comments
 (0)