ValidationError with with_structured_output and Gemini when optional Pydantic fields (Type | None) are omitted by LLM

[CloudMathias]

When using langchain_google_genai models (e.g., Gemini) with with_structured_output and a Pydantic model containing optional fields (defined using Type | None), a ValidationError occurs if the Gemini model omits the optional field in its output rather than explicitly setting it to None.

Pydantic v2 expects fields to be present (even if None) unless explicitly configured otherwise. It appears the integration layer for with_structured_output with Gemini passes the raw dictionary output (e.g., {'sex': 'male'}) directly to the Pydantic model, which fails validation if an optional field like age (defined as int | None) is missing entirely from the dictionary.

This issue seems specific to the Gemini integration, other LLM are working fine and Gemini embedded structured output works with this pydantic model.

Here is the structure I want the LLM to use:

class AgeAndSex(BaseModel):
    age: int | None = Field(description="The age of the user, if known")
    sex: Literal["male", "female"] | None = Field(description="The sex of the user, if known")
...

SYSTEM_PROMPT = "Given what the user has said during the conversation, determine their age and their biological sex (not gender)"

...
structured_llm = llm.with_structured_output(AgeAndSex)

If the LLM is missing one information from the user, it won't be able to return None has a value and we'll return the below error:

messages = [
            AIMessage("Hi. How can I help you?"),
            HumanMessage("I have a headache"),
            AIMessage("Are you male or female? and what is your age"),
            HumanMessage("I am Male"),
]

/usr/local/lib/python3.10/dist-packages/pydantic/main.py in init(self, **data)
212 # __tracebackhide__ tells pytest and some other tools to omit this function from tracebacks
213 tracebackhide = True
--> 214 validated_self = self.pydantic_validator.validate_python(data, self_instance=self)
215 if self is not validated_self:
216 warnings.warn(

ValidationError: 1 validation error for AgeAndSex
age
Field required [type=missing, input_value={'sex': 'male'}, input_type=dict]
For further information visit https://errors.pydantic.dev/2.10/v/missing