Skip to content

FallbackModel does not support Native structured outputs for models supporting it #3104

@bver

Description

@bver

Initial Checks

Description

The Model which supports native structured outputs (output_type=NativeOutput(...)) fails when used from the FallbackModel.

I see that:

  • the 1st agent.run_sync works
  • the 2nd agent.run_sync raises UserError('Native structured output is not supported by this model.')

I would expect both agent.run_sync calls working in this case.

I think it is fine if there is a model in the fallback chain that does not support structured output -- however it should fail only if it is actually supposed to be used.

Example Code

from pydantic import BaseModel, Field
from pydantic_ai import Agent, NativeOutput
from pydantic_ai.providers.azure import AzureProvider
from pydantic_ai.models.fallback import FallbackModel
from pydantic_ai.models.openai import OpenAIChatModel


provider = AzureProvider(azure_endpoint="[redacted]", api_version="2025-03-01-preview")
model = OpenAIChatModel('gpt-4', provider=provider)

class SummaryResult(BaseModel):
    name: str = Field(
        description="Short summary for naming the answer in less than 100 characters.",
        max_length=100,
    )
    summary: str = Field(
        description="Longer description, the answer",
        max_length=1000,
    )

print("working with Model:")
agent = Agent(model, output_type=NativeOutput(SummaryResult, strict=True))
result = agent.run_sync('Where does "hello world" come from?')  
print(result)

print("\nfailing with FallbackModel:")
fallback_model = FallbackModel(model)
agent = Agent(
    fallback_model,
    output_type=NativeOutput(SummaryResult, strict=True),  # this is not supported
)
agent.run_sync('Where does "hello world" come from?')

Python, Pydantic AI & LLM client version

Python version: 3.13.7
Pydantic AI version: 1.0.15
LLM: OpenAIChatModel over AzureProvider

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions