Skip to content

Toolsets #2024

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 91 commits into
base: main
Choose a base branch
from
Draft

Toolsets #2024

wants to merge 91 commits into from
+2,892 −980

Conversation

DouweM
Copy link
Contributor

@DouweM DouweM commented Jun 19, 2025 •
edited
Loading

To fix:

  • MCPToolset
  • Retry counting
  • Streaming tool calls

To improve:

  • Name conflict error messages and logic
  • Performance, caching when possible
  • Considering making MCPServer a toolset, instead of needing MCPServer.as_toolset()
  • Various other TODO comments
  • How output tools are handled
  • DeferredToolCalls with run_stream

To add:

DouweM added 30 commits June 3, 2025 03:20
@DouweM
WIP: Output modes
e290951
@DouweM
WIP: More output modes
2056539
@DouweM
Merge remote-tracking branch 'origin/main' into output-modes
bceba19 
# Conflicts:
#	tests/models/test_openai.py
@DouweM
Fix tests
0cb25c4
@DouweM
Remove syntax invalid before Python 3.12
933b74e
@DouweM
Fix tests
7974df0
@DouweM
Add TextOutput marker
9cc19e2
@DouweM
Merge remote-tracking branch 'origin/main' into output-modes
bc6bb65 
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/_output.py
#	pydantic_ai_slim/pydantic_ai/models/openai.py
#	tests/models/test_openai.py
@DouweM
Add VCR recording of new test
0e356a3
@DouweM
Implement additional output modes in GeminiModel and GoogleModel
81312dc
@DouweM
Fix prompted_json on OpenAIResponses
52ef4d5
@DouweM
Test output modes on Gemini and Anthropic
fe05956
@DouweM
Add VCR recordings of Gemini output mode tests
94421f3
@DouweM
Remove some old TODO comments
1902d00
@DouweM
Add missing VCR recording of Gemini output mode test
1f53c9b
@DouweM
Add more missing VCR recordings
a4c2877
@DouweM
Fix OpenAI tools
56e58f9
@DouweM
Improve test coverage
a5234e1
@DouweM
Update unsupported output mode error message
40def08
@DouweM
Improve test coverage
837d305
@DouweM
Merge branch 'main' into output-modes
3598bef
@DouweM
Test streaming with structured text output
5f71ba8
@DouweM
Make TextOutputFunction Python 3.9 compatible
cfc2749
@DouweM
Properly merge JSON schemas accounting for defs
a137641
@DouweM
Refactor output schemas and modes: more 'isinstance(output_schema, ..…
f495d46 
….)', less 'output_schema.mode == ...'
@DouweM
Merge branch 'main' into output-modes
449ed0d 
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/models/openai.py
#	pydantic_ai_slim/pydantic_ai/profiles/openai.py
#	tests/models/test_google.py
#	tests/models/test_openai_responses.py
@DouweM
Clean up some variable names
e70d249
@DouweM
Improve test coverage
4592b0b
@DouweM
Merge branch 'main' into output-modes
db1c628 
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/_utils.py
#	pydantic_ai_slim/pydantic_ai/agent.py
#	tests/models/test_anthropic.py
#	tests/test_utils.py
@DouweM
Combine JsonSchemaOutput and PromptedJsonOutput into StructuredTextOu…
f57d078 
…tput
DouweM added 2 commits July 2, 2025 00:10
@DouweM
Merge branch 'main' into toolsets
64dacbb 
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/tools.py
@DouweM
Fix docs example
5ca305e
@DouweM DouweM requested review from dmontagu and Copilot July 2, 2025 00:19
Copilot
Copilot AI reviewed Jul 2, 2025
View reviewed changes
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors and extends the toolsets mechanism, consolidating function, output, deferred, and MCP‐based tools under a unified AbstractToolset API, and adds support for:

  • Custom per‐run preparation, prefixing, and processing of tool definitions
  • Deferred tool calls and streaming tool invocation with retry/count tracking
  • Passing toolsets to individual run/iter/run_stream calls, overriding agent‐level toolsets

Reviewed Changes

Copilot reviewed 36 out of 36 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pydantic_ai_slim/pydantic_ai/toolsets/function.py Implements FunctionToolset for registering Python callables as LLM tools.
pydantic_ai_slim/pydantic_ai/toolsets/combined.py Adds CombinedToolset to merge multiple toolsets.
pydantic_ai_slim/pydantic_ai/toolsets/prefixed.py Prefixes tool names per run.
pydantic_ai_slim/pydantic_ai/toolsets/deferred.py Supports deferred tools that end a run.
pydantic_ai_slim/pydantic_ai/mcp.py Migrates MCPServer into the new toolset system.
pydantic_ai_slim/pydantic_ai/output.py Integrates output tools into the new toolset model.
tests/test_toolset.py Adds comprehensive tests for FunctionToolset.
tests/test_tools.py Updates expectations around function toolset lookup and call semantics.
tests/test_streaming.py Aligns streaming tests with the new toolset API.
tests/test_mcp.py Switches to toolsets and call_tool(ctx, …) API.
tests/test_agent.py Reconciles agent‐level toolset construction and naming.

@DouweM
Copy link
Contributor Author

DouweM commented Jul 2, 2025 •
edited
Loading

@claude Review please

Edit: GHA is not installed yet 😄

dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
dmontagu
dmontagu reviewed Jul 2, 2025
View reviewed changes
DouweM added 4 commits July 2, 2025 22:43
@DouweM
Address some feedback
c5ef5f6
@DouweM
Merge branch 'main' into toolsets
badbe23 
# Conflicts:
#	pydantic_ai_slim/pydantic_ai/agent.py
#	pydantic_ai_slim/pydantic_ai/mcp.py
@DouweM
Add sampling_model to Agent __init__, iter, run (etc), and override, …
acddb8d 
…pass sampling_model to MCPServer through RunContext, and make Agent an async contextmanager instead of run_toolsets
@DouweM
Turn RunContext.retries from a defaultdict into a dict again as the 0…
89fc266 
… being stored on read broke a test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dmontagu dmontagu dmontagu left review comments

Copilot code review Copilot Copilot left review comments

Assignees

@DouweM DouweM

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make output_type be conditionally available Toolset, take 3
2 participants
@DouweM @dmontagu