Skip to content

docs[patch]: Update structured output docs to be more opinionated #5968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jul 3, 2024
Merged

docs[patch]: Update structured output docs to be more opinionated #5968

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
01425c8
Update structured output docs to be more opinionated
jacoblee93 Jul 3, 2024
cab3ab8
Typo
jacoblee93 Jul 3, 2024
76d4ed5
Fix
jacoblee93 Jul 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 50 additions & 7 deletions docs/core_docs/docs/concepts.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -821,14 +821,56 @@ a few ways to get structured output from models in LangChain.

#### `.withStructuredOutput()`

For convenience, some LangChain chat models support a `.withStructuredOutput()` method.
For convenience, some LangChain chat models support a [`.withStructuredOutput()`](/docs/how_to/structured_output/#the-.withstructuredoutput-method) method.
This method only requires a schema as input, and returns an object matching the requested schema.
Generally, this method is only present on models that support one of the more advanced methods described below,
and will use one of them under the hood. It takes care of importing a suitable output parser and
formatting the schema in the right format for the model.

Here's an example:

```ts
import { z } from "zod";

const joke = z.object({
setup: z.string().describe("The setup of the joke"),
punchline: z.string().describe("The punchline to the joke"),
rating: z.number().optional().describe("How funny the joke is, from 1 to 10"),
});

// Can also pass in JSON schema.
// It's also beneficial to pass in an additional "name" parameter to give the
// model more context around the type of output to generate.
const structuredLlm = model.withStructuredOutput(joke);

await structuredLlm.invoke("Tell me a joke about cats");
```

```
{
setup: "Why don't cats play poker in the wild?",
punchline: "Too many cheetahs.",
rating: 7
}
```

We recommend this method as a starting point when working with structured output:

- It uses other model-specific features under the hood, without the need to import an output parser.
- For the models that use tool calling, no special prompting is needed.
- If multiple underlying techniques are supported, you can supply a `method` parameter to
[toggle which one is used](/docs/how_to/structured_output/#specifying-the-output-method-advanced).

You may want or need to use other techiniques if:

- The chat model you are using does not support tool calling.
- You are working with very complex schemas and the model is having trouble generating outputs that conform.

For more information, check out this [how-to guide](/docs/how_to/structured_output/#the-.withstructuredoutput-method).

You can also check out [this table](/docs/integrations/chat/) for a list of models that support
`.withStructuredOutput()`.

#### Raw prompting

The most intuitive way to get a model to structure output is to ask nicely.
Expand All @@ -851,9 +893,8 @@ However, there are some drawbacks too:
Some may be better at interpreting [JSON schema](https://json-schema.org/), others may be best with TypeScript definitions,
and still others may prefer XML.

While we'll next go over some ways that you can take advantage of features offered by
model providers to increase reliability, prompting techniques remain important for tuning your
results no matter what method you choose.
While features offered by model providers may increase reliability, prompting techniques remain important for tuning your
results no matter which method you choose.

#### JSON mode

Expand All @@ -864,10 +905,12 @@ Some models, such as [Mistral](/docs/integrations/chat/mistral/), [OpenAI](/docs
support a feature called **JSON mode**, usually enabled via config.

When enabled, JSON mode will constrain the model's output to always be some sort of valid JSON.
Often they require some custom prompting, but it's usually much less burdensome and along the lines of,
`"you must always return JSON"`, and the [output is easier to parse](/docs/how_to/output_parser_json/).
Often they require some custom prompting, but it's usually much less burdensome than completely raw prompting and
more along the lines of,
`"you must always return JSON"`. The [output also is generally easier to parse](/docs/how_to/output_parser_json/).

It's also generally simpler and more commonly available than tool calling.
It's also generally simpler to use directly and more commonly available than tool calling, and can give
more flexibility around prompting and shaping results than tool calling.

Here's an example:

Expand Down
135 changes: 113 additions & 22 deletions docs/core_docs/docs/how_to/structured_output.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,9 +70,9 @@
"import { z } from \"zod\";\n",
"\n",
"const joke = z.object({\n",
" setup: z.string().describe(\"The setup of the joke\"),\n",
" punchline: z.string().describe(\"The punchline to the joke\"),\n",
" rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
" setup: z.string().describe(\"The setup of the joke\"),\n",
" punchline: z.string().describe(\"The punchline to the joke\"),\n",
" rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
"});\n",
"\n",
"const structuredLlm = model.withStructuredOutput(joke);\n",
Expand Down Expand Up @@ -153,22 +153,22 @@
],
"source": [
"const structuredLlm = model.withStructuredOutput(\n",
" {\n",
" \"name\": \"joke\",\n",
" \"description\": \"Joke to tell user.\",\n",
" \"parameters\": {\n",
" \"title\": \"Joke\",\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"setup\": {\"type\": \"string\", \"description\": \"The setup for the joke\"},\n",
" \"punchline\": {\"type\": \"string\", \"description\": \"The joke's punchline\"},\n",
" },\n",
" \"required\": [\"setup\", \"punchline\"],\n",
" },\n",
" }\n",
" {\n",
" \"name\": \"joke\",\n",
" \"description\": \"Joke to tell user.\",\n",
" \"parameters\": {\n",
" \"title\": \"Joke\",\n",
" \"type\": \"object\",\n",
" \"properties\": {\n",
" \"setup\": {\"type\": \"string\", \"description\": \"The setup for the joke\"},\n",
" \"punchline\": {\"type\": \"string\", \"description\": \"The joke's punchline\"},\n",
" },\n",
" \"required\": [\"setup\", \"punchline\"],\n",
" },\n",
" }\n",
")\n",
"\n",
"await structuredLlm.invoke(\"Tell me a joke about cats\")"
"await structuredLlm.invoke(\"Tell me a joke about cats\", { name: \"joke\" })"
]
},
{
Expand Down Expand Up @@ -213,24 +213,115 @@
],
"source": [
"const structuredLlm = model.withStructuredOutput(joke, {\n",
" method: \"json_mode\",\n",
" name: \"joke\",\n",
" method: \"json_mode\",\n",
" name: \"joke\",\n",
"})\n",
"\n",
"await structuredLlm.invoke(\n",
" \"Tell me a joke about cats, respond in JSON with `setup` and `punchline` keys\"\n",
" \"Tell me a joke about cats, respond in JSON with `setup` and `punchline` keys\"\n",
")"
]
},
{
"cell_type": "markdown",
"id": "5e92a98a",
"id": "56278a82",
"metadata": {},
"source": [
"In the above example, we use OpenAI's alternate JSON mode capability along with a more specific prompt.\n",
"\n",
"For specifics about the model you choose, peruse its entry in the [API reference pages](https://v02.api.js.langchain.com/).\n",
"For specifics about the model you choose, peruse its entry in the [API reference pages](https://api.js.langchain.com/).\n",
"\n",
"### (Advanced) Raw outputs\n",
"\n",
"LLMs aren't perfect at generating structured output, especially as schemas become complex. You can avoid raising exceptions and handle the raw output yourself by passing `includeRaw: true`. This changes the output format to contain the raw message output and the `parsed` value (if successful):"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "46b616a4",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{\n",
" raw: AIMessage {\n",
" lc_serializable: \u001b[33mtrue\u001b[39m,\n",
" lc_kwargs: {\n",
" content: \u001b[32m\"\"\u001b[39m,\n",
" tool_calls: [\n",
" {\n",
" name: \u001b[32m\"joke\"\u001b[39m,\n",
" args: \u001b[36m[Object]\u001b[39m,\n",
" id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m\n",
" }\n",
" ],\n",
" invalid_tool_calls: [],\n",
" additional_kwargs: { function_call: \u001b[90mundefined\u001b[39m, tool_calls: [ \u001b[36m[Object]\u001b[39m ] },\n",
" response_metadata: {}\n",
" },\n",
" lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
" content: \u001b[32m\"\"\u001b[39m,\n",
" name: \u001b[90mundefined\u001b[39m,\n",
" additional_kwargs: {\n",
" function_call: \u001b[90mundefined\u001b[39m,\n",
" tool_calls: [\n",
" {\n",
" id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m,\n",
" type: \u001b[32m\"function\"\u001b[39m,\n",
" function: \u001b[36m[Object]\u001b[39m\n",
" }\n",
" ]\n",
" },\n",
" response_metadata: {\n",
" tokenUsage: { completionTokens: \u001b[33m33\u001b[39m, promptTokens: \u001b[33m88\u001b[39m, totalTokens: \u001b[33m121\u001b[39m },\n",
" finish_reason: \u001b[32m\"stop\"\u001b[39m\n",
" },\n",
" tool_calls: [\n",
" {\n",
" name: \u001b[32m\"joke\"\u001b[39m,\n",
" args: {\n",
" setup: \u001b[32m\"Why was the cat sitting on the computer?\"\u001b[39m,\n",
" punchline: \u001b[32m\"Because it wanted to keep an eye on the mouse!\"\u001b[39m,\n",
" rating: \u001b[33m7\u001b[39m\n",
" },\n",
" id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m\n",
" }\n",
" ],\n",
" invalid_tool_calls: [],\n",
" usage_metadata: { input_tokens: \u001b[33m88\u001b[39m, output_tokens: \u001b[33m33\u001b[39m, total_tokens: \u001b[33m121\u001b[39m }\n",
" },\n",
" parsed: {\n",
" setup: \u001b[32m\"Why was the cat sitting on the computer?\"\u001b[39m,\n",
" punchline: \u001b[32m\"Because it wanted to keep an eye on the mouse!\"\u001b[39m,\n",
" rating: \u001b[33m7\u001b[39m\n",
" }\n",
"}"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"const joke = z.object({\n",
" setup: z.string().describe(\"The setup of the joke\"),\n",
" punchline: z.string().describe(\"The punchline to the joke\"),\n",
" rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
"});\n",
"\n",
"const structuredLlm = model.withStructuredOutput(joke, { includeRaw: true, name: \"joke\" });\n",
"\n",
"await structuredLlm.invoke(\"Tell me a joke about cats\");"
]
},
{
"cell_type": "markdown",
"id": "5e92a98a",
"metadata": {},
"source": [
"## Prompting techniques\n",
"\n",
"You can also prompt models to outputting information in a given format. This approach relies on designing good prompts and then parsing the output of the models. This is the only option for models that don't support `.with_structured_output()` or other built-in approaches.\n",
Expand Down
Loading