Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f5911d3
Support enhanced JSON Schema features in Google Gemini 2.5+ models
conradlee Nov 6, 2025
083b369
Document discriminator field limitation and add test
conradlee Nov 6, 2025
270c8dd
Fix discriminator test and update with proper type annotations
conradlee Nov 6, 2025
78b174b
Fix tests and docs: Enhanced features only work with Vertex AI
conradlee Nov 6, 2025
46690c5
Create separate transformers for Vertex AI and GLA
conradlee Nov 6, 2025
98c24f4
Merge branch 'main' into feat/google-enhanced-json-schema
conradlee Nov 12, 2025
16871ef
remove verbose documentation of minor update
conradlee Nov 12, 2025
3e07326
Address PR review: Use response_json_schema and simplify implementation
conradlee Nov 12, 2025
2d9ea0d
Remove simplify_nullable_unions - Google supports type: 'null' natively
conradlee Nov 12, 2025
1bfaad9
Add tests for enhanced JSON Schema features and remove enum conversion
conradlee Nov 12, 2025
88be4f8
Test gemini-2.5-flash recursive schemas with Vertex AI (passes)
conradlee Nov 12, 2025
5a1faf7
Remove unnecessary __init__ override in GoogleJsonSchemaTransformer
conradlee Nov 12, 2025
b1d6433
Fix test failures: update snapshots for native JSON Schema support
conradlee Nov 12, 2025
d054438
Fix Vertex AI cassette: correct project ID and content-length
conradlee Nov 12, 2025
3c022cc
Merge branch 'main' into feat/google-enhanced-json-schema
DouweM Nov 12, 2025
dd63c0b
Update pydantic_ai_slim/pydantic_ai/models/google.py
DouweM Nov 12, 2025
36b2e38
Merge branch 'main' into feat/google-enhanced-json-schema
conradlee Nov 13, 2025
02a8231
Address maintainer review: fix comment typo and add prefixItems test
conradlee Nov 13, 2025
db5968a
Add cassette for prefixItems test
conradlee Nov 13, 2025
9254fd5
Remove dead code: simplify_nullable_unions feature
conradlee Nov 13, 2025
c760141
Remove additional dead code: single-member union collapse path
conradlee Nov 13, 2025
c977993
Merge branch 'main' into feat/google-enhanced-json-schema
conradlee Nov 13, 2025
2b9e2d0
JsonSchemaTransformer: Add back support for the simplify_nullable_uni…
conradlee Nov 14, 2025
7c00439
Merge branch 'main' into feat/google-enhanced-json-schema
conradlee Nov 14, 2025
8dcf07a
cosmetic changes
conradlee Nov 14, 2025
8611b4f
Add test confirming gemini-2.5-flash recursive schema support on GLA
conradlee Nov 15, 2025
a27e8f2
Update Vertex AI recursive schema test cassette
conradlee Nov 15, 2025
091c605
Merge branch 'main' into feat/google-enhanced-json-schema
conradlee Nov 15, 2025
bc2447d
Fix content-length header after JWT token scrubbing
conradlee Nov 15, 2025
e640bda
Fix Vertex AI cassette to use CI environment values
conradlee Nov 17, 2025
75c3e8d
Fix flaky test_a2a_error_handling test
conradlee Nov 17, 2025
46d6e09
Move test_simplify_nullable_unions to dedicated test_json_schema module
conradlee Nov 18, 2025
9bb1763
Remove redundant Vertex AI recursive schema test
conradlee Nov 18, 2025
e9230c2
Merge remote-tracking branch 'origin/main' into feat/google-enhanced-…
conradlee Nov 18, 2025
d3bb3c8
Add comprehensive gemini-2.0-flash tests for enhanced JSON schema fea…
conradlee Nov 18, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions pydantic_ai_slim/pydantic_ai/_json_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ def __init__(
*,
strict: bool | None = None,
prefer_inlined_defs: bool = False,
simplify_nullable_unions: bool = False,
simplify_nullable_unions: bool = False, # TODO (v2): Remove this, no longer used
):
self.schema = schema

Expand Down Expand Up @@ -146,10 +146,9 @@ def _handle_union(self, schema: JsonSchema, union_kind: Literal['anyOf', 'oneOf'

handled = [self._handle(member) for member in members]

# convert nullable unions to nullable types
# TODO (v2): Remove this feature, no longer used
if self.simplify_nullable_unions:
handled = self._simplify_nullable_union(handled)

if len(handled) == 1:
# In this case, no need to retain the union
return handled[0] | schema
Expand All @@ -161,7 +160,7 @@ def _handle_union(self, schema: JsonSchema, union_kind: Literal['anyOf', 'oneOf'

@staticmethod
def _simplify_nullable_union(cases: list[JsonSchema]) -> list[JsonSchema]:
# TODO: Should we move this to relevant subclasses? Or is it worth keeping here to make reuse easier?
# TODO (v2): Remove this method, no longer used
if len(cases) == 2 and {'type': 'null'} in cases:
# Find the non-null schema
non_null_schema = next(
Expand Down
4 changes: 2 additions & 2 deletions pydantic_ai_slim/pydantic_ai/models/google.py
Original file line number Diff line number Diff line change
Expand Up @@ -292,7 +292,7 @@ async def count_tokens(
thinking_config=generation_config.get('thinking_config'),
media_resolution=generation_config.get('media_resolution'),
response_mime_type=generation_config.get('response_mime_type'),
response_schema=generation_config.get('response_schema'),
response_json_schema=generation_config.get('response_json_schema'),
),
)

Expand Down Expand Up @@ -456,7 +456,7 @@ async def _build_content_and_config(
tools=cast(ToolListUnionDict, tools),
tool_config=tool_config,
response_mime_type=response_mime_type,
response_schema=response_schema,
response_json_schema=response_schema,
response_modalities=modalities,
)
return contents, config
Expand Down
70 changes: 5 additions & 65 deletions pydantic_ai_slim/pydantic_ai/profiles/google.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
from __future__ import annotations as _annotations

import warnings

from pydantic_ai.exceptions import UserError

from .._json_schema import JsonSchema, JsonSchemaTransformer
from . import ModelProfile

Expand All @@ -23,84 +19,28 @@ def google_model_profile(model_name: str) -> ModelProfile | None:
class GoogleJsonSchemaTransformer(JsonSchemaTransformer):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@conradlee We're so close but I found another wrinkle: the PR title says "Gemini 2.5+", so I thought I'd check whether there's any version gating to do to keep Gemini 2.0 working, and it appears there is:

https://ai.google.dev/gemini-api/docs/structured-output?example=recipe#model_support says:

Note that Gemini 2.0 requires an explicit propertyOrdering list within the JSON input to define the preferred structure.

google-gemini/cookbook@fb674d1#diff-74f587f27880bc1ab4e63ebe4dac03066cd9826ac77282fa1101aeec9ae29825R426 says:

Important: Gemini 2.0 models require explicit ordering of keys in structured output schemas. When working with Gemini 2.0, you must define the desired property ordering as a list within the propertyOrdering field as part of your schema configuration.

So we need a test for Gemini 2.0, and I expect that in the google_model_profile function here, we need to use a slightly different JSON schema transformer for 2.0, possible a subclass that does all the same thing + set propertyOrdering (unless it turns out it's not actually required).

Copy link
Contributor Author

@conradlee conradlee Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@DouweM You're right to be sceptical of whether this will work on gemini 2.0. However I remember when I was setting the tests up some of them first pointed to gemini-2.0-flash and passed.

To that end I had a suspicion that gemini-2.0-flash supports all these features too--even if the documentation says it does not.

I have thus added versions of the tests that point to gemini-2.5-flash as well. They all pass. I think this is good news.

"""Transforms the JSON Schema from Pydantic to be suitable for Gemini.
Gemini which [supports](https://ai.google.dev/gemini-api/docs/function-calling#function_declarations)
a subset of OpenAPI v3.0.3.
Specifically:
* gemini doesn't allow the `title` keyword to be set
* gemini doesn't allow `$defs` — we need to inline the definitions where possible
Gemini supports [a subset of OpenAPI v3.0.3](https://ai.google.dev/gemini-api/docs/function-calling#function_declarations).
"""

def __init__(self, schema: JsonSchema, *, strict: bool | None = None):
super().__init__(schema, strict=strict, prefer_inlined_defs=True, simplify_nullable_unions=True)

def transform(self, schema: JsonSchema) -> JsonSchema:
# Note: we need to remove `additionalProperties: False` since it is currently mishandled by Gemini
additional_properties = schema.pop(
'additionalProperties', None
) # don't pop yet so it's included in the warning
if additional_properties:
original_schema = {**schema, 'additionalProperties': additional_properties}
warnings.warn(
'`additionalProperties` is not supported by Gemini; it will be removed from the tool JSON schema.'
f' Full schema: {self.schema}\n\n'
f'Source of additionalProperties within the full schema: {original_schema}\n\n'
'If this came from a field with a type like `dict[str, MyType]`, that field will always be empty.\n\n'
"If Google's APIs are updated to support this properly, please create an issue on the Pydantic AI GitHub"
' and we will fix this behavior.',
UserWarning,
)

schema.pop('title', None)
# Remove properties not supported by Gemini
schema.pop('$schema', None)
if (const := schema.pop('const', None)) is not None:
# Gemini doesn't support const, but it does support enum with a single value
schema['enum'] = [const]
schema.pop('discriminator', None)
schema.pop('examples', None)

# TODO: Should we use the trick from pydantic_ai.models.openai._OpenAIJsonSchema
# where we add notes about these properties to the field description?
schema.pop('exclusiveMaximum', None)
schema.pop('exclusiveMinimum', None)

# Gemini only supports string enums, so we need to convert any enum values to strings.
# Pydantic will take care of transforming the transformed string values to the correct type.
if enum := schema.get('enum'):
schema['type'] = 'string'
schema['enum'] = [str(val) for val in enum]

type_ = schema.get('type')
if 'oneOf' in schema and 'type' not in schema: # pragma: no cover
# This gets hit when we have a discriminated union
# Gemini returns an API error in this case even though it says in its error message it shouldn't...
# Changing the oneOf to an anyOf prevents the API error and I think is functionally equivalent
schema['anyOf'] = schema.pop('oneOf')

if type_ == 'string' and (fmt := schema.pop('format', None)):
description = schema.get('description')
if description:
schema['description'] = f'{description} (format: {fmt})'
else:
schema['description'] = f'Format: {fmt}'

if '$ref' in schema:
raise UserError(f'Recursive `$ref`s in JSON Schema are not supported by Gemini: {schema["$ref"]}')

if 'prefixItems' in schema:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have a test yet that verifies that prefixItems now works

Copy link
Contributor Author

@conradlee conradlee Nov 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now added a test for this based on a coordinate class whose json schema representation looks like

{
  "description": "A 2D coordinate with latitude and longitude.",
  "properties": {
    "point": {
      "maxItems": 2,
      "minItems": 2,
      "prefixItems": [
        {
          "type": "number"
        },
        {
          "type": "number"
        }
      ],
      "title": "Point",
      "type": "array"
    }
  },
  "required": [
    "point"
  ],
  "title": "Coordinate",
  "type": "object"
}

Luckily this test passes with the google provider.

# prefixItems is not currently supported in Gemini, so we convert it to items for best compatibility
prefix_items = schema.pop('prefixItems')
items = schema.get('items')
unique_items = [items] if items is not None else []
for item in prefix_items:
if item not in unique_items:
unique_items.append(item)
if len(unique_items) > 1: # pragma: no cover
schema['items'] = {'anyOf': unique_items}
elif len(unique_items) == 1: # pragma: no branch
schema['items'] = unique_items[0]
schema.setdefault('minItems', len(prefix_items))
if items is None: # pragma: no branch
schema.setdefault('maxItems', len(prefix_items))
# Note: exclusiveMinimum/exclusiveMaximum are NOT yet supported
schema.pop('exclusiveMinimum', None)
schema.pop('exclusiveMaximum', None)

return schema
2 changes: 1 addition & 1 deletion tests/json_body_serializer.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def serialize(cassette_dict: Any): # pragma: lax no cover
del data['body']
if content_type == ['application/x-www-form-urlencoded']:
query_params = urllib.parse.parse_qs(data['body'])
for key in ['client_secret', 'refresh_token']: # pragma: no cover
for key in ['client_id', 'client_secret', 'refresh_token']: # pragma: no cover
if key in query_params:
query_params[key] = ['scrubbed']
data['body'] = urllib.parse.urlencode(query_params)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
interactions:
- request:
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '519'
content-type:
- application/json
host:
- generativelanguage.googleapis.com
method: POST
parsed_body:
contents:
- parts:
- text: Create a config named "api-config" with metadata author="Alice" and version="1.0"
role: user
generationConfig:
responseJsonSchema:
description: A response with configuration metadata.
properties:
metadata:
additionalProperties:
type: string
type: object
name:
type: string
required:
- name
- metadata
title: ConfigResponse
type: object
responseMimeType: application/json
responseModalities:
- TEXT
uri: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent
response:
headers:
alt-svc:
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length:
- '631'
content-type:
- application/json; charset=UTF-8
server-timing:
- gfet4t7; dur=1379
transfer-encoding:
- chunked
vary:
- Origin
- X-Origin
- Referer
parsed_body:
candidates:
- content:
parts:
- text: '{"name": "api-config", "metadata": {"author": "Alice", "version": "1.0"}}'
role: model
finishReason: STOP
index: 0
modelVersion: gemini-2.5-flash
responseId: CZMUacOtKv2SxN8Pi7TrsAs
usageMetadata:
candidatesTokenCount: 25
promptTokenCount: 23
promptTokensDetails:
- modality: TEXT
tokenCount: 23
thoughtsTokenCount: 158
totalTokenCount: 206
status:
code: 200
message: OK
version: 1
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
interactions:
- request:
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '519'
content-type:
- application/json
host:
- generativelanguage.googleapis.com
method: POST
parsed_body:
contents:
- parts:
- text: Create a config named "api-config" with metadata author="Alice" and version="1.0"
role: user
generationConfig:
responseJsonSchema:
description: A response with configuration metadata.
properties:
metadata:
additionalProperties:
type: string
type: object
name:
type: string
required:
- name
- metadata
title: ConfigResponse
type: object
responseMimeType: application/json
responseModalities:
- TEXT
uri: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent
response:
headers:
alt-svc:
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
content-length:
- '760'
content-type:
- application/json; charset=UTF-8
server-timing:
- gfet4t7; dur=818
transfer-encoding:
- chunked
vary:
- Origin
- X-Origin
- Referer
parsed_body:
candidates:
- avgLogprobs: -1.4634492981713265e-05
content:
parts:
- text: |-
{
"name": "api-config",
"metadata": {
"author": "Alice",
"version": "1.0"
}
}
role: model
finishReason: STOP
modelVersion: gemini-2.0-flash
responseId: 8pMcab_EMqWd28oP46bOiAk
usageMetadata:
candidatesTokenCount: 40
candidatesTokensDetails:
- modality: TEXT
tokenCount: 40
promptTokenCount: 22
promptTokensDetails:
- modality: TEXT
tokenCount: 22
totalTokenCount: 62
status:
code: 200
message: OK
version: 1
Loading