Skip to content

with_structured_output() does not work with Gemini 2.5 Models #40

@MeerrettichMeister

Description

@MeerrettichMeister

Using structured output with the new Gemini models dont work.

from langchain_core.messages import HumanMessage
from langchain_oci import ChatOCIGenAI
from pydantic import BaseModel


class Formatted(BaseModel):
    answer: str
    user_request: str


oci_chat = ChatOCIGenAI(
    model_id="google.gemini-2.5-flash",
    service_endpoint="https://inference.generativeai.us-ashburn-1.oci.oraclecloud.com",
    compartment_id="compartment_id",
)


unstructured_response = oci_chat.invoke(
    [HumanMessage("Tell me something about the statue of liberty")]
)

structured_chat = oci_chat.with_structured_output(Formatted)
structured_response = structured_chat.invoke(
    [HumanMessage("Tell me something about the statue of liberty")]
)
print(unstructured_response)
print(structured_response)

Running the above code returns a proper response with the regular chat interface but fails with the structured output.
It either returns None or fails by the pydantic validation.

Python 3.12
langchain 0.3.27
langchain-oci 0.1.5
pydantic 2.11.7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions