initial thinking support for claude #15092

eneufeld · 2025-03-01T23:38:49Z

What it does

Updated the Anthropic SDK from v0.32.1 to v0.39.0
Added support for thinking messages in the language model interface
Added a ThinkingChatResponseContent implementation
Added proper handling for Claude's thinking feature in the Anthropic language model
Added a TextPartRenderer to display thinking content
Various interface updates to support thinking messages throughout the code

How to test

Add this or similar to your settings.json:

"ai-features.modelSettings.requestSettings": [
    {
        "scope": {
            "modelId": "claude-3-7-sonnet-latest",
            "agentId": "Architect"
        },
        "requestSettings": {
            "max_tokens": 40000,
            "temperature": 1,
            "thinking": {
                "type": "enabled",
                "budget_tokens": 8192
            }
        }
    }
],

This will enable the thinking mode on the Architect.
As you can provide chat settings you can just add the requestSettings part to your chat for any agent as long as it uses sonnet-3.7

Follow-ups

Breaking changes

This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

JonasHelming · 2025-03-12T15:20:32Z

@eneufeld This is currently under work, right? So no review needed right now?

eneufeld · 2025-03-12T16:12:25Z

exactly, no review needed, a test would be nice to confirm we go into the right direction. Then we can discuss next steps what needs to be added to make this final

Currently, our application has fragmented tracking of the LLM communications: - Chat model doesn't track all LLM messages (e.g. system message) or multiple LLM requests per user response - Chat agents must manually record LLM communications via `CommunicationRecordingService` - Tool calls are tracked in the chat model in content parts, which are rather UI focused, but tool calls missing in the `CommunicationRecordingService` This fragmentation means we lack a single source of truth for chat session information. This PR centralizes all LLM communication tracking in the chat model by: 1. Introducing a new `ChatLanguageModelService` that encapsulates LLM request tracking 2. Moving related code from the abstract chat agent to this new service 3. Feeding the `CommunicationRecordingService` via a simple listener on the chat model - Complete tracking of all LLM requests, responses, and tool calls in one place - Simplified interface for chat agents to interact with LLMs - Improved data structure to capture tool calls and LLM messages in the History View (with minor UI enhancements) Contributed on behalf of STMicroelectronics

* Move default request settings from individual models to frontend LanguageModelService * Enhance RequestSettingsSchema with model/provider/agent scoping * Add client-side settings for tool calls and thinking messages

- Updated the Anthropic SDK from v0.32.1 to v0.39.0 - Added support for thinking messages in the language model interface - Added a ThinkingChatResponseContent implementation - Added proper handling for Claude's thinking feature in the Anthropic language model - Added a TextPartRenderer to display thinking content - Various interface updates to support thinking messages throughout the code

sdirix

I had a first look at the PR. To enable work on the PR I leave this intermediate review and will come with a full review later.

Overall it looks very good. However I did not yet

actually run the code
looked into detail on how the new request settings determination works

packages/ai-anthropic/src/node/anthropic-language-model.ts

packages/ai-chat-ui/src/browser/chat-response-renderer/thinking-part-renderer.tsx

packages/ai-chat-ui/src/browser/chat-view-widget-toolbar-contribution.tsx

packages/ai-core/src/common/language-model-service.ts

packages/ai-hugging-face/src/node/huggingface-language-model.ts

packages/ai-openai/src/node/openai-language-model.ts

packages/ai-terminal/src/browser/ai-terminal-agent.ts

sdirix

I had a closer look at the PR. You can find my comments inline.

I tested the thinking support via the request settings in the preferences, as well as via the chat session requests. Both worked for me 👍

Based on my last review you adapted the handling of empty message for the HuggingFace language model, however we should handle this the same way for all language models. So we should either revert that change or apply it to all of them.

packages/ai-chat-ui/src/browser/chat-response-renderer/thinking-part-renderer.tsx

sdirix · 2025-03-26T06:59:14Z

packages/ai-chat-ui/src/browser/chat-view-commands.ts

+    export const EDIT_SESSION_SETTINGS = Command.toLocalizedCommand({
+        id: 'chat:widget:session-settings',
+        category: CHAT_CATEGORY,
+        iconClass: codicon('bracket')


Would be nice if this would not be a static icon, but update depending on whether the user did some customization. Similar to the history view sort button which changes depending on the sort state.

If this is out of scope for this PR then please create a follow up ticket to track this work.

added to the ai epic

packages/ai-core/src/browser/ai-core-preferences.ts

packages/ai-core/src/browser/frontend-language-model-service.ts

packages/ai-core/src/browser/ai-core-preferences.ts

packages/ai-core/src/browser/frontend-language-model-service.ts

sdirix · 2025-03-26T10:23:29Z

packages/ai-chat-ui/src/browser/chat-response-renderer/thinking-part-renderer.tsx

+            <div className='theia-thinking'>
+                <details>
+                    <summary>{nls.localize('theia/ai/chat-ui/thinking-part-renderer/thinking', 'Thinking')}</summary>
+                    <pre>{response.content}</pre>


I would prefer this to be regular text or markdown rendering. Wrapping in <pre> does not do any wrapping, making it painful to inspect for longer lines.

I added text wrapping for the pre element, maybe this is enough

I think the rendering can be further improved, but this can be done in a follow up

sdirix

Works for me! Great work ❤️

I only have minor comments left (the unresolved ones). Feel free to tackle them here or to create follow up issue(s).

sdirix · 2025-03-26T12:54:44Z

@eneufeld actually this is a breaking change. So we should at adapt the changelog and at least indicate the changes in the request API

CHANGELOG.md

eneufeld requested review from planger and JonasHelming March 1, 2025 23:38

github-project-automation bot added this to PR Backlog Mar 1, 2025

github-project-automation bot moved this to Waiting on reviewers in PR Backlog Mar 1, 2025

eneufeld mentioned this pull request Mar 4, 2025

Theia AI - Epic - next #15068

Open

66 tasks

eneufeld force-pushed the feat/claude-thinking branch 2 times, most recently from 333d23b to 3d50d23 Compare March 15, 2025 01:08

eneufeld mentioned this pull request Mar 17, 2025

feat: centralize LLM communication tracking in chat model #15146

Closed

2 tasks

planger and others added 9 commits March 25, 2025 00:22

Add LLM request to terminal agent recording

028d0e4

Add LLM requests to code completion agent

c47ca40

let all request go through LanguageModelService, mode it to ai-core

385be02

centralize request settings handling

e6d559d

* Move default request settings from individual models to frontend LanguageModelService * Enhance RequestSettingsSchema with model/provider/agent scoping * Add client-side settings for tool calls and thinking messages

rebase on top of 15146

2486f81

fix tests

140212f

add session settings

f33edbb

eneufeld force-pushed the feat/claude-thinking branch from 1e2270a to f33edbb Compare March 24, 2025 23:41

sdirix requested changes Mar 25, 2025

View reviewed changes

github-project-automation bot moved this from Waiting on reviewers to Waiting on author in PR Backlog Mar 25, 2025

review fixes

31046ae

sdirix requested changes Mar 26, 2025

View reviewed changes

sdirix reviewed Mar 26, 2025

View reviewed changes

packages/ai-core/src/browser/ai-core-preferences.ts Show resolved Hide resolved

more review improvements

e185d5d

sdirix requested changes Mar 26, 2025

View reviewed changes

sdirix marked this pull request as ready for review March 26, 2025 10:24

more review comment improvements

389f2e0

sdirix approved these changes Mar 26, 2025

View reviewed changes

github-project-automation bot moved this from Waiting on author to Needs merge in PR Backlog Mar 26, 2025

some more review fixes

139018f

sdirix reviewed Mar 26, 2025

View reviewed changes

CHANGELOG.md Show resolved Hide resolved

sdirix approved these changes Mar 26, 2025

View reviewed changes

sdirix merged commit 4877a0b into master Mar 27, 2025
10 of 11 checks passed

github-project-automation bot moved this from Needs merge to Done in PR Backlog Mar 27, 2025

sdirix deleted the feat/claude-thinking branch March 27, 2025 16:08

github-actions bot added this to the 1.60.0 milestone Mar 27, 2025

eneufeld mentioned this pull request Mar 27, 2025

[Theia AI] Add LLMSettings to UI #15100

Closed

JonasHelming mentioned this pull request Apr 5, 2025

Theia AI Epic - 1.60 #15405

Closed

37 tasks

xpomul mentioned this pull request Apr 16, 2025

Every request to ollama ends up in "Cannot read properties of undefined (reading 'createMessageContent')" #15475

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

initial thinking support for claude #15092

initial thinking support for claude #15092

eneufeld commented Mar 1, 2025 •

edited by sdirix

Loading

JonasHelming commented Mar 12, 2025

eneufeld commented Mar 12, 2025

sdirix left a comment

sdirix left a comment

sdirix Mar 26, 2025

eneufeld Mar 26, 2025

sdirix Mar 26, 2025

eneufeld Mar 26, 2025

sdirix Mar 26, 2025

sdirix left a comment

sdirix commented Mar 26, 2025

initial thinking support for claude #15092

initial thinking support for claude #15092

Conversation

eneufeld commented Mar 1, 2025 • edited by sdirix Loading

What it does

How to test

Follow-ups

Breaking changes

Attribution

Review checklist

Reminder for reviewers

JonasHelming commented Mar 12, 2025

eneufeld commented Mar 12, 2025

sdirix left a comment

Choose a reason for hiding this comment

sdirix left a comment

Choose a reason for hiding this comment

sdirix Mar 26, 2025

Choose a reason for hiding this comment

eneufeld Mar 26, 2025

Choose a reason for hiding this comment

sdirix Mar 26, 2025

Choose a reason for hiding this comment

eneufeld Mar 26, 2025

Choose a reason for hiding this comment

sdirix Mar 26, 2025

Choose a reason for hiding this comment

sdirix left a comment

Choose a reason for hiding this comment

sdirix commented Mar 26, 2025

eneufeld commented Mar 1, 2025 •

edited by sdirix

Loading