docs[patch]: Update structured output docs to be more opinionated (#5968)

jacoblee93 · web-flow · commit d0bc53ac21a4 · 2024-07-02T21:22:37.000-07:00
* Update structured output docs to be more opinionated

* Typo

* Fix
diff --git a/docs/core_docs/docs/concepts.mdx b/docs/core_docs/docs/concepts.mdx
@@ -821,14 +821,56 @@ a few ways to get structured output from models in LangChain.
 
 #### `.withStructuredOutput()`
 
-For convenience, some LangChain chat models support a `.withStructuredOutput()` method.
+For convenience, some LangChain chat models support a [`.withStructuredOutput()`](/docs/how_to/structured_output/#the-.withstructuredoutput-method) method.
 This method only requires a schema as input, and returns an object matching the requested schema.
 Generally, this method is only present on models that support one of the more advanced methods described below,
 and will use one of them under the hood. It takes care of importing a suitable output parser and
 formatting the schema in the right format for the model.
 
+Here's an example:
+
+```ts
+import { z } from "zod";
+
+const joke = z.object({
+  setup: z.string().describe("The setup of the joke"),
+  punchline: z.string().describe("The punchline to the joke"),
+  rating: z.number().optional().describe("How funny the joke is, from 1 to 10"),
+});
+
+// Can also pass in JSON schema.
+// It's also beneficial to pass in an additional "name" parameter to give the
+// model more context around the type of output to generate.
+const structuredLlm = model.withStructuredOutput(joke);
+
+await structuredLlm.invoke("Tell me a joke about cats");
+```
+
+```
+{
+  setup: "Why don't cats play poker in the wild?",
+  punchline: "Too many cheetahs.",
+  rating: 7
+}
+```
+
+We recommend this method as a starting point when working with structured output:
+
+- It uses other model-specific features under the hood, without the need to import an output parser.
+- For the models that use tool calling, no special prompting is needed.
+- If multiple underlying techniques are supported, you can supply a `method` parameter to
+  [toggle which one is used](/docs/how_to/structured_output/#specifying-the-output-method-advanced).
+
+You may want or need to use other techiniques if:
+
+- The chat model you are using does not support tool calling.
+- You are working with very complex schemas and the model is having trouble generating outputs that conform.
+
 For more information, check out this [how-to guide](/docs/how_to/structured_output/#the-.withstructuredoutput-method).
 
+You can also check out [this table](/docs/integrations/chat/) for a list of models that support
+`.withStructuredOutput()`.
+
 #### Raw prompting
 
 The most intuitive way to get a model to structure output is to ask nicely.
@@ -851,9 +893,8 @@ However, there are some drawbacks too:
   Some may be better at interpreting [JSON schema](https://json-schema.org/), others may be best with TypeScript definitions,
   and still others may prefer XML.
 
-While we'll next go over some ways that you can take advantage of features offered by
-model providers to increase reliability, prompting techniques remain important for tuning your
-results no matter what method you choose.
+While features offered by model providers may increase reliability, prompting techniques remain important for tuning your
+results no matter which method you choose.
 
 #### JSON mode
 
@@ -864,10 +905,12 @@ Some models, such as [Mistral](/docs/integrations/chat/mistral/), [OpenAI](/docs
 support a feature called **JSON mode**, usually enabled via config.
 
 When enabled, JSON mode will constrain the model's output to always be some sort of valid JSON.
-Often they require some custom prompting, but it's usually much less burdensome and along the lines of,
-`"you must always return JSON"`, and the [output is easier to parse](/docs/how_to/output_parser_json/).
+Often they require some custom prompting, but it's usually much less burdensome than completely raw prompting and
+more along the lines of,
+`"you must always return JSON"`. The [output also is generally easier to parse](/docs/how_to/output_parser_json/).
 
-It's also generally simpler and more commonly available than tool calling.
+It's also generally simpler to use directly and more commonly available than tool calling, and can give
+more flexibility around prompting and shaping results than tool calling.
 
 Here's an example:
 
diff --git a/docs/core_docs/docs/how_to/structured_output.ipynb b/docs/core_docs/docs/how_to/structured_output.ipynb
@@ -70,9 +70,9 @@
     "import { z } from \"zod\";\n",
     "\n",
     "const joke = z.object({\n",
-    "    setup: z.string().describe(\"The setup of the joke\"),\n",
-    "    punchline: z.string().describe(\"The punchline to the joke\"),\n",
-    "    rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
+    "  setup: z.string().describe(\"The setup of the joke\"),\n",
+    "  punchline: z.string().describe(\"The punchline to the joke\"),\n",
+    "  rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
     "});\n",
     "\n",
     "const structuredLlm = model.withStructuredOutput(joke);\n",
@@ -153,22 +153,22 @@
    ],
    "source": [
     "const structuredLlm = model.withStructuredOutput(\n",
-    "    {\n",
-    "        \"name\": \"joke\",\n",
-    "        \"description\": \"Joke to tell user.\",\n",
-    "        \"parameters\": {\n",
-    "            \"title\": \"Joke\",\n",
-    "            \"type\": \"object\",\n",
-    "            \"properties\": {\n",
-    "                \"setup\": {\"type\": \"string\", \"description\": \"The setup for the joke\"},\n",
-    "                \"punchline\": {\"type\": \"string\", \"description\": \"The joke's punchline\"},\n",
-    "            },\n",
-    "            \"required\": [\"setup\", \"punchline\"],\n",
-    "        },\n",
-    "    }\n",
+    "  {\n",
+    "    \"name\": \"joke\",\n",
+    "    \"description\": \"Joke to tell user.\",\n",
+    "    \"parameters\": {\n",
+    "      \"title\": \"Joke\",\n",
+    "      \"type\": \"object\",\n",
+    "      \"properties\": {\n",
+    "        \"setup\": {\"type\": \"string\", \"description\": \"The setup for the joke\"},\n",
+    "        \"punchline\": {\"type\": \"string\", \"description\": \"The joke's punchline\"},\n",
+    "      },\n",
+    "      \"required\": [\"setup\", \"punchline\"],\n",
+    "    },\n",
+    "  }\n",
     ")\n",
     "\n",
-    "await structuredLlm.invoke(\"Tell me a joke about cats\")"
+    "await structuredLlm.invoke(\"Tell me a joke about cats\", { name: \"joke\" })"
    ]
   },
   {
@@ -213,24 +213,115 @@
    ],
    "source": [
     "const structuredLlm = model.withStructuredOutput(joke, {\n",
-    "    method: \"json_mode\",\n",
-    "    name: \"joke\",\n",
+    "  method: \"json_mode\",\n",
+    "  name: \"joke\",\n",
     "})\n",
     "\n",
     "await structuredLlm.invoke(\n",
-    "    \"Tell me a joke about cats, respond in JSON with `setup` and `punchline` keys\"\n",
+    "  \"Tell me a joke about cats, respond in JSON with `setup` and `punchline` keys\"\n",
     ")"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5e92a98a",
+   "id": "56278a82",
    "metadata": {},
    "source": [
     "In the above example, we use OpenAI's alternate JSON mode capability along with a more specific prompt.\n",
     "\n",
-    "For specifics about the model you choose, peruse its entry in the [API reference pages](https://v02.api.js.langchain.com/).\n",
+    "For specifics about the model you choose, peruse its entry in the [API reference pages](https://api.js.langchain.com/).\n",
     "\n",
+    "### (Advanced) Raw outputs\n",
+    "\n",
+    "LLMs aren't perfect at generating structured output, especially as schemas become complex. You can avoid raising exceptions and handle the raw output yourself by passing `includeRaw: true`. This changes the output format to contain the raw message output and the `parsed` value (if successful):"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "46b616a4",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "{\n",
+       "  raw: AIMessage {\n",
+       "    lc_serializable: \u001b[33mtrue\u001b[39m,\n",
+       "    lc_kwargs: {\n",
+       "      content: \u001b[32m\"\"\u001b[39m,\n",
+       "      tool_calls: [\n",
+       "        {\n",
+       "          name: \u001b[32m\"joke\"\u001b[39m,\n",
+       "          args: \u001b[36m[Object]\u001b[39m,\n",
+       "          id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m\n",
+       "        }\n",
+       "      ],\n",
+       "      invalid_tool_calls: [],\n",
+       "      additional_kwargs: { function_call: \u001b[90mundefined\u001b[39m, tool_calls: [ \u001b[36m[Object]\u001b[39m ] },\n",
+       "      response_metadata: {}\n",
+       "    },\n",
+       "    lc_namespace: [ \u001b[32m\"langchain_core\"\u001b[39m, \u001b[32m\"messages\"\u001b[39m ],\n",
+       "    content: \u001b[32m\"\"\u001b[39m,\n",
+       "    name: \u001b[90mundefined\u001b[39m,\n",
+       "    additional_kwargs: {\n",
+       "      function_call: \u001b[90mundefined\u001b[39m,\n",
+       "      tool_calls: [\n",
+       "        {\n",
+       "          id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m,\n",
+       "          type: \u001b[32m\"function\"\u001b[39m,\n",
+       "          function: \u001b[36m[Object]\u001b[39m\n",
+       "        }\n",
+       "      ]\n",
+       "    },\n",
+       "    response_metadata: {\n",
+       "      tokenUsage: { completionTokens: \u001b[33m33\u001b[39m, promptTokens: \u001b[33m88\u001b[39m, totalTokens: \u001b[33m121\u001b[39m },\n",
+       "      finish_reason: \u001b[32m\"stop\"\u001b[39m\n",
+       "    },\n",
+       "    tool_calls: [\n",
+       "      {\n",
+       "        name: \u001b[32m\"joke\"\u001b[39m,\n",
+       "        args: {\n",
+       "          setup: \u001b[32m\"Why was the cat sitting on the computer?\"\u001b[39m,\n",
+       "          punchline: \u001b[32m\"Because it wanted to keep an eye on the mouse!\"\u001b[39m,\n",
+       "          rating: \u001b[33m7\u001b[39m\n",
+       "        },\n",
+       "        id: \u001b[32m\"call_0pEdltlfSXjq20RaBFKSQOeF\"\u001b[39m\n",
+       "      }\n",
+       "    ],\n",
+       "    invalid_tool_calls: [],\n",
+       "    usage_metadata: { input_tokens: \u001b[33m88\u001b[39m, output_tokens: \u001b[33m33\u001b[39m, total_tokens: \u001b[33m121\u001b[39m }\n",
+       "  },\n",
+       "  parsed: {\n",
+       "    setup: \u001b[32m\"Why was the cat sitting on the computer?\"\u001b[39m,\n",
+       "    punchline: \u001b[32m\"Because it wanted to keep an eye on the mouse!\"\u001b[39m,\n",
+       "    rating: \u001b[33m7\u001b[39m\n",
+       "  }\n",
+       "}"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "const joke = z.object({\n",
+    "  setup: z.string().describe(\"The setup of the joke\"),\n",
+    "  punchline: z.string().describe(\"The punchline to the joke\"),\n",
+    "  rating: z.number().optional().describe(\"How funny the joke is, from 1 to 10\"),\n",
+    "});\n",
+    "\n",
+    "const structuredLlm = model.withStructuredOutput(joke, { includeRaw: true, name: \"joke\" });\n",
+    "\n",
+    "await structuredLlm.invoke(\"Tell me a joke about cats\");"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5e92a98a",
+   "metadata": {},
+   "source": [
     "## Prompting techniques\n",
     "\n",
     "You can also prompt models to outputting information in a given format. This approach relies on designing good prompts and then parsing the output of the models. This is the only option for models that don't support `.with_structured_output()` or other built-in approaches.\n",
