Skip to content

docs[minor]: bedrock tool calling/WSO docs #5735

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions docs/core_docs/docs/integrations/chat/bedrock.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -56,3 +56,31 @@ Anthropic Claude-3 models hosted on Bedrock have multimodal capabilities and can
import BedrockMultimodalExample from "@examples/models/chat/integration_bedrock_multimodal.ts";

<CodeBlock language="typescript">{BedrockMultimodalExample}</CodeBlock>

### Tool calling

:::info
Not all Bedrock models support tool calling. Please refer to the [model documentation](https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html) for more information.
:::

The examples below demonstrate how to use tool calling, along with the `withStructuredOutput` method to easily compose structured output LLM calls.

import ToolCalling from "@examples/models/chat/integration_bedrock_tools.ts";

<CodeBlock language="typescript">{ToolCalling}</CodeBlock>

:::tip
See the LangSmith trace [here](https://smith.langchain.com/public/003a684d-90eb-406e-a146-8ee5e617921b/r)
:::

#### `.withStructuredOutput({ ... })`

Using the `.withStructuredOutput` method, you can easily make the LLM return structured output, given only a Zod or JSON schema:

import WSOExample from "@examples/models/chat/integration_bedrock_wso.ts";

<CodeBlock language="typescript">{WSOExample}</CodeBlock>

:::tip
See the LangSmith trace [here](https://smith.langchain.com/public/1f7b1ad8-e4ac-4965-8ce1-fae06005f3d7/r)
:::
63 changes: 63 additions & 0 deletions examples/src/models/chat/integration_bedrock_tools.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! I've reviewed the code and flagged a specific change related to accessing environment variables using process.env. This is to ensure the maintainers review and confirm the handling of environment variables in the code. Let me know if you need further assistance!

// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";

const model = new BedrockChat({
region: process.env.BEDROCK_AWS_REGION,
model: "anthropic.claude-3-sonnet-20240229-v1:0",
maxRetries: 0,
credentials: {
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
},
});

const weatherSchema = z
.object({
city: z.string().describe("The city to get the weather for"),
state: z.string().describe("The state to get the weather for").optional(),
})
.describe("Get the weather for a city");

const modelWithTools = model.bindTools([
{
name: "weather_tool",
description: weatherSchema.description,
input_schema: zodToJsonSchema(weatherSchema),
},
]);
// Optionally, you can bind tools via the `.bind` method:
// const modelWithTools = model.bind({
// tools: [
// {
// name: "weather_tool",
// description: weatherSchema.description,
// input_schema: zodToJsonSchema(weatherSchema),
// },
// ],
// });

const res = await modelWithTools.invoke("What's the weather in New York?");
console.log(res);

/*
AIMessage {
additional_kwargs: { id: 'msg_bdrk_01JF7hb4PNQPywP4gnBbgpHi' },
response_metadata: {
stop_reason: 'tool_use',
usage: { input_tokens: 300, output_tokens: 85 }
},
tool_calls: [
{
name: 'weather_tool',
args: {
city: 'New York',
state: 'NY'
},
id: 'toolu_bdrk_01AtEZRTCKioFXqhoNcpgaV7'
}
],
}
*/
34 changes: 34 additions & 0 deletions examples/src/models/chat/integration_bedrock_wso.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there! 👋 I've reviewed the code changes and noticed that environment variables are being accessed using process.env. I've flagged this for your review to ensure that the handling of environment variables aligns with best practices. Let me know if you have any questions!

// Or, from web environments:
// import { BedrockChat } from "@langchain/community/chat_models/bedrock/web";
import { z } from "zod";

const model = new BedrockChat({
region: process.env.BEDROCK_AWS_REGION,
model: "anthropic.claude-3-sonnet-20240229-v1:0",
maxRetries: 0,
credentials: {
secretAccessKey: process.env.BEDROCK_AWS_SECRET_ACCESS_KEY!,
accessKeyId: process.env.BEDROCK_AWS_ACCESS_KEY_ID!,
},
});

const weatherSchema = z
.object({
city: z.string().describe("The city to get the weather for"),
state: z.string().describe("The state to get the weather for").optional(),
})
.describe("Get the weather for a city");

const modelWithStructuredOutput = model.withStructuredOutput(weatherSchema, {
name: "weather_tool", // Optional, defaults to 'extract'
});

const res = await modelWithStructuredOutput.invoke(
"What's the weather in New York?"
);
console.log(res);

/*
{ city: 'New York', state: 'NY' }
*/
Loading