Skip to content

feat: add CerebrasProvider #1867

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

smallstepman
Copy link

@smallstepman smallstepman commented May 29, 2025

as discussed in #1826
based on GrokProvider from #1842

json schema transformer compatibility

I found out setting simplify_nullable_unions to False improves the compatibility, as it circumvents the following error: pydantic_ai.exceptions.ModelHTTPError: status_code: 400, model_name: qwen-3-32b, body: {'message': "Unsupported JSON schema fields: {'nullable'}", 'type': 'invalid_request_error', 'param': 'response_format', 'code': 'wrong_api_format'}

I had to manually set it for

class GoogleJsonSchemaTransformer(JsonSchemaTransformer):
    def __init__(self, schema: JsonSchema, *, strict: bool | None = None):
        super().__init__(schema, strict=strict, prefer_inlined_defs=True, simplify_nullable_unions=False)

let me know if I can either:

  • expose simplify_nullable_unions via __init__ params with default set to True (current value in upstream), or
  • copy-paste-rename whole GoogleJsonSchemaTransformer

test with

import os

from pydantic import BaseModel, Field

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from pydantic_ai.providers.cerebras import CerebrasProvider

cerebras_provider = CerebrasProvider(api_key=os.environ.get('CEREBRAS_API_KEY'))


class SomeModelNestedNestedNestedNested(BaseModel):
    greet_me: str


class SomeModelNestedNestedNested(BaseModel):
    nested: SomeModelNestedNestedNestedNested
    value: int | None


class SomeModelNestedNested(BaseModel):
    nested: SomeModelNestedNestedNested


class SomeModelNested(BaseModel):
    nested: SomeModelNestedNested


class SomeModel(BaseModel):
    text: str
    something_good: str | int | None | float | list[str] = Field(default=None)
    something_null: None
    nested: SomeModelNested


for m in [
    'llama-4-scout-17b-16e-instruct',
    'llama3.1-8b',
    'llama-3.3-70b',
    'qwen-3-32b',
    'deepseek-r1-distill-llama-70b',
]:
    model = OpenAIModel(m, provider=cerebras_provider)
    agent = Agent(model, output_type=SomeModel)
    print(agent.run_sync('hello'))

@smallstepman smallstepman marked this pull request as draft May 29, 2025 22:57
@smallstepman
Copy link
Author

smallstepman commented May 29, 2025

hitting

❯ make lint     
uv run ruff format --check
219 files already formatted
uv run ruff check
pydantic_ai_slim/pydantic_ai/providers/__init__.py:51:5: C901 `infer_provider` is too complex (16 > 15)
   |
51 | def infer_provider(provider: str) -> Provider[Any]:
   |     ^^^^^^^^^^^^^^ C901
52 |     """Infer the provider from the provider name."""
53 |     if provider == 'openai':
   |

Found 1 error.
make: *** [lint] Error 1

let me know how you'd like to solve this

mccabe = { max-complexity = 15 }

Copy link
Contributor

hyperlint-ai bot commented May 29, 2025

PR Change Summary

Added the CerebrasProvider to enhance API integration capabilities.

  • Introduced CerebrasProvider for API integration.
  • Updated documentation to include usage examples for CerebrasProvider.
  • Provided instructions for obtaining an API key from Cerebras.

Modified Files

  • docs/api/providers.md
  • docs/models/openai.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

@smallstepman smallstepman marked this pull request as ready for review May 29, 2025 23:10
def client(self) -> AsyncOpenAI:
return self._client

def model_profile(self, model_name: str) -> ModelProfile | None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should do automatic model profile selection here based on the underlying model used, like we do in TogetherProvider. We can still override the json_schema_transformer unconditionally, like we do there as well.

Copy link
Author

@smallstepman smallstepman Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit puzzled on

        provider, model_name = model_name.split('/', 1)
        if provider in provider_to_profile:
            profile = provider_to_profile[provider](model_name)

in TogetherProvider.

I dont fully grasp the idea behind what's meant to be a "provider" in this case. Should CerebrasProvider support alternative model name syntax like Qwen/Qwen-3-32b instead of what's listed in their docs: qwen-3-32b?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smallstepman What we're trying to do is to use the regular model names that Cerebras expects to choose the right model profile for the model in question, based on the prefix. In the case of TogetherProvider their model names have a <provider>/ prefix, but for example in GroqProvider you can see how we match on a simple qwen prefix and then use qwen_model_provider. We'll want to do the same here for all models Cerebras supports that match models we already have model profiles for.

@DouweM
Copy link
Contributor

DouweM commented Jun 3, 2025

I found out setting simplify_nullable_unions to False improves the compatibility, as it circumvents the following error

@smallstepman Is this Qwen specific or do all models used with Cerebras need this? Either way we'll want a new subclass as ModelProfile take a type, not an instance (so we can't pass a different value to __init__).

@DouweM
Copy link
Contributor

DouweM commented Jun 3, 2025

let me know how you'd like to solve this

@smallstepman You can add # noqa: C901 at the end of the def line to ignore it in this case.

@smallstepman
Copy link
Author

smallstepman commented Jun 3, 2025

I found out setting simplify_nullable_unions to False improves the compatibility, as it circumvents the following error

@smallstepman Is this Qwen specific or do all models used with Cerebras need this? Either way we'll want a new subclass as ModelProfile take a type, not an instance (so we can't pass a different value to __init__).

it's weird, but 75% of the models need this (the only exception is llama-3.3-70b which is capable of producing structured output even when simplify_nullable_unions is set to True). With simplify_nullable_unions=False, 100% of models work alright with the nested structured output test case I've been using.

@smallstepman smallstepman requested a review from DouweM June 3, 2025 19:44
@smallstepman
Copy link
Author

please advise on how to solve failing coverage check

def __init__(self, schema: JsonSchema, *, strict: bool | None = None):
super().__init__(schema, strict=strict, prefer_inlined_defs=True, simplify_nullable_unions=False)

def transform(self, schema: JsonSchema) -> JsonSchema:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we please reduce the duplication here with the GoogleJsonSchemaTransformer, perhaps by subclassing it?

def client(self) -> AsyncOpenAI:
return self._client

def model_profile(self, model_name: str) -> ModelProfile | None:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smallstepman What we're trying to do is to use the regular model names that Cerebras expects to choose the right model profile for the model in question, based on the prefix. In the case of TogetherProvider their model names have a <provider>/ prefix, but for example in GroqProvider you can see how we match on a simple qwen prefix and then use qwen_model_provider. We'll want to do the same here for all models Cerebras supports that match models we already have model profiles for.

@DouweM
Copy link
Contributor

DouweM commented Jun 6, 2025

please advise on how to solve failing coverage check

@smallstepman When we remove the duplicated code in the json schema transformer by inheriting from one of the existing ones instead, this issue should go away!

Co-authored-by: Douwe Maan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants