Incorrect max_tokens in yaml file for AWS Bedrock US Cross Region Inference version of 3.5 Sonnet v2 and 3.5 Haiku

[kazuhisa-wada]

Self Checks

• This is only for bug report, if you would like to ask a question, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Dify version

0.11.2

Cloud or Self Hosted

Cloud, Self Hosted (Docker), Self Hosted (Source)

Steps to reproduce

For below models, max tokens configuration in each yaml is incorrect and then max token can be configured up to 4096 on UI, although it should be set up to 8192. This partially seemed to have been fixed in this pr #10286 but the cross region cases were not covered.

• Bedrock Claude 3.5 Sonnet V2 US Cross Region Inference
• Bedrock Claude 3.5 Haiku US Cross Region Inference

✔️ Expected Behavior

On UI, max tokens can be set to 8192 at maximum for

• Bedrock Claude 3.5 Sonnet V2 US Cross Region Inference
• Bedrock Claude 3.5 Haiku US Cross Region Inference

❌ Actual Behavior

On UI, max tokens can be set to 4096 at maximum for

• Bedrock Claude 3.5 Sonnet V2 US Cross Region Inference
• Bedrock Claude 3.5 Haiku US Cross Region Inference

[dosubot]

Hello @kazuhisa-wada! I'm here to help you with any bugs, questions, or contributions you have for the project. Let me know how I can assist you!

I found a similar issue related to the max tokens configuration for AWS Bedrock models. The problem is that the max_tokens parameter in the YAML file is set with a maximum value of 4096, which is incorrect for the models you mentioned. The configuration should allow a maximum of 8192 tokens. This issue was partially addressed in a pull request, but the cross-region cases were not covered ^[1].

To resolve this, you need to update the max_tokens parameter in the YAML configuration files for the affected models. Change the max value from 4096 to 8192 to reflect the correct maximum token limit. This will ensure that the UI allows setting the max tokens to 8192 as expected.

To continue talking to Dosu, mention @dosu.