Skip to content

Commit 09f6af2

Browse files
jacoblee93jacoblee93
authored
docs[patch]: Adds structured output section to concepts (#5750)
* Adds structured output section to concepts * Fix links * Fix * Update table * Fix * Update table
1 parent 4f3deb2 commit 09f6af2

File tree

3 files changed

+134
-31
lines changed

3 files changed

+134
-31
lines changed

‎docs/core_docs/docs/concepts.mdx

+108-6
Original file line numberDiff line numberDiff line change
@@ -144,7 +144,7 @@ Chat models support the assignment of distinct roles to conversation messages, h
144144

145145
Although the underlying models are messages in, message out, the LangChain wrappers also allow these models to take a string as input.
146146
This gives them the same interface as LLMs (and simpler to use).
147-
When a string is passed in as input, it will be converted to a HumanMessage under the hood before being passed to the underlying model.
147+
When a string is passed in as input, it will be converted to a `HumanMessage` under the hood before being passed to the underlying model.
148148

149149
LangChain does not host any Chat Models, rather we rely on third party integrations.
150150

@@ -751,7 +751,105 @@ You can roughly think of it as an iterator over callback events (though the form
751751

752752
See [this guide](/docs/how_to/streaming/#using-stream-events) for more detailed information on how to use `.streamEvents()`.
753753

754-
### Function/tool calling
754+
### Structured output
755+
756+
LLMs are capable of generating arbitrary text. This enables the model to respond appropriately to a wide
757+
range of inputs, but for some use-cases, it can be useful to constrain the LLM's output
758+
to a specific format or structure. This is referred to as **structured output**.
759+
760+
For example, if the output is to be stored in a relational database,
761+
it is much easier if the model generates output that adheres to a defined schema or format.
762+
[Extracting specific information](/docs/tutorials/extraction/) from unstructured text is another
763+
case where this is particularly useful. Most commonly, the output format will be JSON,
764+
though other formats such as [XML](/docs/how_to/output_parser_xml/) can be useful too. Below, we'll discuss
765+
a few ways to get structured output from models in LangChain.
766+
767+
#### `.withStructuredOutput()`
768+
769+
For convenience, some LangChain chat models support a `.withStructuredOutput()` method.
770+
This method only requires a schema as input, and returns an object matching the requested schema.
771+
Generally, this method is only present on models that support one of the more advanced methods described below,
772+
and will use one of them under the hood. It takes care of importing a suitable output parser and
773+
formatting the schema in the right format for the model.
774+
775+
For more information, check out this [how-to guide](/docs/how_to/structured_output/#the-.withstructuredoutput-method).
776+
777+
#### Raw prompting
778+
779+
The most intuitive way to get a model to structure output is to ask nicely.
780+
In addition to your query, you can give instructions describing what kind of output you'd like, then
781+
parse the output using an [output parser](/docs/concepts/#output-parsers) to convert the raw
782+
model message or string output into something more easily manipulated.
783+
784+
The biggest benefit to raw prompting is its flexibility:
785+
786+
- Raw prompting does not require any special model features, only sufficient reasoning capability to understand
787+
the passed schema.
788+
- You can prompt for any format you'd like, not just JSON. This can be useful if the model you
789+
are using is more heavily trained on a certain type of data, such as XML or YAML.
790+
791+
However, there are some drawbacks too:
792+
793+
- LLMs are non-deterministic, and prompting a LLM to consistently output data in the exactly correct format
794+
for smooth parsing can be surprisingly difficult and model-specific.
795+
- Individual models have quirks depending on the data they were trained on, and optimizing prompts can be quite difficult.
796+
Some may be better at interpreting [JSON schema](https://json-schema.org/), others may be best with TypeScript definitions,
797+
and still others may prefer XML.
798+
799+
While we'll next go over some ways that you can take advantage of features offered by
800+
model providers to increase reliability, prompting techniques remain important for tuning your
801+
results no matter what method you choose.
802+
803+
#### JSON mode
804+
805+
<span data-heading-keywords="json mode"></span>
806+
807+
Some models, such as [Mistral](/docs/integrations/chat/mistral/), [OpenAI](/docs/integrations/chat/openai/),
808+
[Together AI](/docs/integrations/chat/togetherai/) and [Ollama](/docs/integrations/chat/ollama/),
809+
support a feature called **JSON mode**, usually enabled via config.
810+
811+
When enabled, JSON mode will constrain the model's output to always be some sort of valid JSON.
812+
Often they require some custom prompting, but it's usually much less burdensome and along the lines of,
813+
`"you must always return JSON"`, and the [output is easier to parse](/docs/how_to/output_parser_json/).
814+
815+
It's also generally simpler and more commonly available than tool calling.
816+
817+
Here's an example:
818+
819+
```ts
820+
import { JsonOutputParser } from "@langchain/core/output_parsers";
821+
import { ChatPromptTemplate } from "@langchain/core/prompts";
822+
import { ChatOpenAI } from "@langchain/openai";
823+
824+
const model = new ChatOpenAI({
825+
model: "gpt-4o",
826+
modelKwargs: {
827+
response_format: { type: "json_object" },
828+
},
829+
});
830+
831+
const TEMPLATE = `Answer the user's question to the best of your ability.
832+
You must always output a JSON object with an "answer" key and a "followup_question" key.
833+
834+
{question}`;
835+
836+
const prompt = ChatPromptTemplate.fromTemplate(TEMPLATE);
837+
838+
const chain = prompt.pipe(model).pipe(new JsonOutputParser());
839+
840+
await chain.invoke({ question: "What is the powerhouse of the cell?" });
841+
```
842+
843+
```
844+
{
845+
answer: "The powerhouse of the cell is the mitochondrion.",
846+
followup_question: "Would you like to learn more about the functions of mitochondria?"
847+
}
848+
```
849+
850+
For a full list of model providers that support JSON mode, see [this table](/docs/integrations/chat/).
851+
852+
#### Function/tool calling
755853

756854
:::info
757855
We use the term tool calling interchangeably with function calling. Although
@@ -769,8 +867,10 @@ from unstructured text, you could give the model an "extraction" tool that takes
769867
parameters matching the desired schema, then treat the generated output as your final
770868
result.
771869

772-
A tool call includes a name, arguments dict, and an optional identifier. The
773-
arguments dict is structured `{argument_name: argument_value}`.
870+
For models that support it, tool calling can be very convenient. It removes the
871+
guesswork around how best to prompt schemas in favor of a built-in model feature. It can also
872+
more naturally support agentic flows, since you can just pass multiple tool schemas instead
873+
of fiddling with enums or unions.
774874

775875
Many LLM providers, including [Anthropic](https://www.anthropic.com/),
776876
[Cohere](https://cohere.com/), [Google](https://cloud.google.com/vertex-ai),
@@ -787,14 +887,16 @@ LangChain provides a standardized interface for tool calling that is consistent
787887

788888
The standard interface consists of:
789889

790-
- `ChatModel.bindTools()`: a method for specifying which tools are available for a model to call.
890+
- `ChatModel.bindTools()`: a method for specifying which tools are available for a model to call. This method accepts [LangChain tools](/docs/concepts/#tools).
791891
- `AIMessage.toolCalls`: an attribute on the `AIMessage` returned from the model for accessing the tool calls requested by the model.
792892

793-
There are two main use cases for function/tool calling:
893+
The following how-to guides are good practical resources for using function/tool calling:
794894

795895
- [How to return structured data from an LLM](/docs/how_to/structured_output/)
796896
- [How to use a model to call tools](/docs/how_to/tool_calling/)
797897

898+
For a full list of model providers that support tool calling, [see this table](/docs/integrations/chat/).
899+
798900
### Retrieval
799901

800902
LangChain provides several advanced retrieval types. A full list is below, along with the following information:

‎docs/core_docs/docs/how_to/structured_output.ipynb

+3
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,9 @@
1616
"metadata": {},
1717
"source": [
1818
"# How to return structured data from a model\n",
19+
"```{=mdx}\n",
20+
"<span data-heading-keywords=\"with_structured_output\"></span>\n",
21+
"```\n",
1922
"\n",
2023
"It is often useful to have a model return output that matches some specific schema. One common use-case is extracting data from arbitrary text to insert into a traditional database or use with some other downstrem system. This guide will show you a few different strategies you can use to do this.\n",
2124
"\n",

‎docs/core_docs/docs/integrations/chat/index.mdx

+23-25
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
---
22
sidebar_position: 1
33
sidebar_class_name: hidden
4+
hide_table_of_contents: true
45
---
56

67
# Chat models
@@ -11,36 +12,33 @@ All ChatModels implement the Runnable interface, which comes with default implem
1112

1213
- _Streaming_ support defaults to returning an `AsyncIterator` of a single value, the final result returned by the underlying ChatModel provider. This obviously doesn't give you token-by-token streaming, which requires native support from the ChatModel provider, but ensures your code that expects an iterator of tokens can work for any of our ChatModel integrations.
1314
- _Batch_ support defaults to calling the underlying ChatModel in parallel for each input. The concurrency can be controlled with the `maxConcurrency` key in `RunnableConfig`.
14-
- _Map_ support defaults to calling `.invoke` across all instances of the array which it was called on.
1515

1616
Each ChatModel integration can optionally provide native implementations to truly enable invoke, streaming or batching requests.
1717

1818
Additionally, some chat models support additional ways of guaranteeing structure in their outputs by allowing you to pass in a defined schema.
19-
[Function calling and parallel function calling](/docs/how_to/tool_calling) (tool calling) are two common ones, and those capabilities allow you to use the chat model as the LLM in certain types of agents.
19+
[Tool calling](/docs/how_to/tool_calling) (tool calling) is one capability, and allows you to use the chat model as the LLM in certain types of agents.
2020
Some models in LangChain have also implemented a `withStructuredOutput()` method that unifies many of these different ways of constraining output to a schema.
2121

2222
The table shows, for each integration, which features have been implemented with native support. Yellow circles (🟡) indicates partial support - for example, if the model supports tool calling but not tool messages for agents.
2323

24-
| Model | Invoke | Stream | Batch | Function Calling | Tool Calling | `withStructuredOutput()` |
25-
| :---------------------- | :----: | :----: | :---: | :--------------: | :-------------------------: | :----------------------: |
26-
| BedrockChat | ✅ | ✅ | ✅ | ❌ | 🟡 (Bedrock Anthropic only) | ❌ |
27-
| ChatAlibabaTongyi | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
28-
| ChatAnthropic | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
29-
| ChatBaiduWenxin | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
30-
| ChatCloudflareWorkersAI | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
31-
| ChatCohere | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
32-
| ChatFireworks | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ |
33-
| ChatGoogleGenerativeAI | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
34-
| ChatGoogleVertexAI | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
35-
| ChatVertexAI | ✅ | ✅ | ✅ | ❌ | ✅ | ✅ |
36-
| ChatGooglePaLM | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
37-
| ChatGroq | ✅ | ✅ | ✅ | ❌ | 🟡 | ✅ |
38-
| ChatLlamaCpp | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
39-
| ChatMinimax | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ |
40-
| ChatMistralAI | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ |
41-
| ChatOllama | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
42-
| ChatOpenAI | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
43-
| ChatTencentHunyuan | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
44-
| ChatTogetherAI | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
45-
| ChatYandexGPT | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
46-
| ChatZhipuAI | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ |
24+
| Model | Stream | JSON mode | [Tool Calling](/docs/how_to/tool_calling/) | [`withStructuredOutput()`](/docs/how_to/structured_output/#the-.withstructuredoutput-method) | [Multimodal](/docs/how_to/multimodal_inputs/) |
25+
| :---------------------- | :----: | :-------: | :----------------------------------------: | :------------------------------------------------------------------------------------------: | :-------------------------------------------: |
26+
| BedrockChat | ✅ | ❌ | 🟡 (Bedrock Anthropic only) | ❌ | ❌ |
27+
| ChatAlibabaTongyi | ❌ | ❌ | ❌ | ❌ | ❌ |
28+
| ChatAnthropic | ✅ | ❌ | ✅ | ✅ | ✅ |
29+
| ChatBaiduWenxin | ❌ | ❌ | ❌ | ❌ | ❌ |
30+
| ChatCloudflareWorkersAI | ✅ | ❌ | ❌ | ❌ | ❌ |
31+
| ChatCohere | ✅ | ❌ | ❌ | ❌ | ❌ |
32+
| ChatFireworks | ✅ | ✅ | ✅ | ❌ | ❌ |
33+
| ChatGoogleGenerativeAI | ✅ | ❌ | ❌ | ❌ | ✅ |
34+
| ChatVertexAI | ✅ | ❌ | ✅ | ✅ | ✅ |
35+
| ChatGroq | ✅ | ✅ | 🟡 | ✅ | ❌ |
36+
| ChatLlamaCpp | ✅ | ❌ | ❌ | ❌ | ❌ |
37+
| ChatMinimax | ❌ | ❌ | ❌ | ❌ | ❌ |
38+
| ChatMistralAI | ❌ | ✅ | ✅ | ✅ | ❌ |
39+
| ChatOllama | ✅ | ✅ | ❌ | ❌ | ❌ |
40+
| ChatOpenAI | ✅ | ✅ | ✅ | ✅ | ✅ |
41+
| ChatTencentHunyuan | ✅ | ❌ | ❌ | ❌ | ❌ |
42+
| ChatTogetherAI | ✅ | ✅ | ❌ | ❌ | ❌ |
43+
| ChatYandexGPT | ❌ | ❌ | ❌ | ❌ | ❌ |
44+
| ChatZhipuAI | ❌ | ❌ | ❌ | ❌ | ❌ |

0 commit comments

Comments
 (0)