Skip to content

Commit 88be4f8

Browse files
conradleeclaude
andcommitted
Test gemini-2.5-flash recursive schemas with Vertex AI (passes)
The test_google_recursive_schema_native_output_gemini_2_5 test now uses vertex_provider and PASSES successfully. NOTE: During development, this test consistently failed with a 500 error when using google_provider (GLA with GEMINI_API_KEY). However, it passes with vertex_provider (Vertex AI). This may be: - A temporary GLA API issue - A limitation specific to certain API keys - An issue with the GLA endpoint for recursive schemas Maintainers should verify this works with their GLA setup before merge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>
1 parent 1bfaad9 commit 88be4f8

File tree

2 files changed

+90
-17
lines changed

2 files changed

+90
-17
lines changed

tests/models/cassettes/test_google/test_google_recursive_schema_native_output_gemini_2_5.yaml

Lines changed: 83 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,49 @@
11
interactions:
2+
- request:
3+
body: grant_type=%5B%27refresh_token%27%5D&client_id=%5B%2732555940559.apps.googleusercontent.com%27%5D&client_secret=%5B%27scrubbed%27%5D&refresh_token=%5B%27scrubbed%27%5D
4+
headers:
5+
accept:
6+
- '*/*'
7+
accept-encoding:
8+
- gzip, deflate
9+
connection:
10+
- keep-alive
11+
content-length:
12+
- '234'
13+
content-type:
14+
- application/x-www-form-urlencoded
15+
method: POST
16+
uri: https://oauth2.googleapis.com/token
17+
response:
18+
headers:
19+
alt-svc:
20+
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
21+
cache-control:
22+
- no-cache, no-store, max-age=0, must-revalidate
23+
content-length:
24+
- '1520'
25+
content-type:
26+
- application/json; charset=utf-8
27+
expires:
28+
- Mon, 01 Jan 1990 00:00:00 GMT
29+
pragma:
30+
- no-cache
31+
transfer-encoding:
32+
- chunked
33+
vary:
34+
- Origin
35+
- X-Origin
36+
- Referer
37+
parsed_body:
38+
access_token: scrubbed
39+
expires_in: 3599
40+
id_token: eyJhbGciOiJSUzI1NiIsImtpZCI6IjRmZWI0NGYwZjdhN2UyN2M3YzQwMzM3OWFmZjIwYWY1YzhjZjUyZGMiLCJ0eXAiOiJKV1QifQ.eyJpc3MiOiJodHRwczovL2FjY291bnRzLmdvb2dsZS5jb20iLCJhenAiOiIzMjU1NTk0MDU1OS5hcHBzLmdvb2dsZXVzZXJjb250ZW50LmNvbSIsImF1ZCI6IjMyNTU1OTQwNTU5LmFwcHMuZ29vZ2xldXNlcmNvbnRlbnQuY29tIiwic3ViIjoiMTA0MDMyODc1Njg3NDUwNzA3NzUwIiwiaGQiOiJjYXB0dXJlZGtub3dsZWRnZS5haSIsImVtYWlsIjoiY29ucmFkQGNhcHR1cmVka25vd2xlZGdlLmFpIiwiZW1haWxfdmVyaWZpZWQiOnRydWUsImF0X2hhc2giOiJ2emc3MEN0a1FhcnBJNVMzYWJZY1ZnIiwiaWF0IjoxNzYyOTU2NjUxLCJleHAiOjE3NjI5NjAyNTF9.P0kjqqgbGDIEfRkaCL76T1rRV1CC6ypQjWLlq8IWDgFhA6xMLOgcoN3eCU0yFg8lgoY_SI2C2oaQWMep9dNZbF4yil376ohzyuxkzyjjjfWmf-IuxDS9_s4IbIOut90XLM_R1SxWA-nc_nrki3OeYbvss0BWh28_BAvYLuMI4EVqW5QnlW1VmYj46kgn80YW9PEwSwei1h99ew9KLg7e9Fhb1LIXdU7zu1NkGjbvygirN3NKEZkry55w2U_h8ItPRes0MqJUFqpJzto92-GtpKhPjbIvmPJfmepxec9Tq-VU5IK24RqmYtNmzT5ZgyOXQtUni-9zhKjWsP8kIbGTEg
41+
scope: openid https://www.googleapis.com/auth/accounts.reauth https://www.googleapis.com/auth/appengine.admin https://www.googleapis.com/auth/cloud-platform
42+
https://www.googleapis.com/auth/userinfo.email https://www.googleapis.com/auth/compute https://www.googleapis.com/auth/sqlservice.login
43+
token_type: Bearer
44+
status:
45+
code: 200
46+
message: OK
247
- request:
348
headers:
449
accept:
@@ -12,7 +57,7 @@ interactions:
1257
content-type:
1358
- application/json
1459
host:
15-
- generativelanguage.googleapis.com
60+
- aiplatform.googleapis.com
1661
method: POST
1762
parsed_body:
1863
contents:
@@ -41,29 +86,56 @@ interactions:
4186
responseMimeType: application/json
4287
responseModalities:
4388
- TEXT
44-
uri: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent
89+
uri: https://aiplatform.googleapis.com/v1beta1/projects/ck-nest-prod/locations/global/publishers/google/models/gemini-2.5-flash:generateContent
4590
response:
4691
headers:
4792
alt-svc:
4893
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
4994
content-length:
50-
- '200'
95+
- '882'
5196
content-type:
5297
- application/json; charset=UTF-8
53-
server-timing:
54-
- gfet4t7; dur=211
5598
transfer-encoding:
5699
- chunked
57100
vary:
58101
- Origin
59102
- X-Origin
60103
- Referer
61104
parsed_body:
62-
error:
63-
code: 500
64-
message: An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting
65-
status: INTERNAL
105+
candidates:
106+
- avgLogprobs: -1.1582396030426025
107+
content:
108+
parts:
109+
- text: |-
110+
{
111+
"value": "A",
112+
"children": [
113+
{
114+
"value": "B"
115+
},
116+
{
117+
"value": "C"
118+
}
119+
]
120+
}
121+
role: model
122+
finishReason: STOP
123+
createTime: '2025-11-12T14:10:52.206764Z'
124+
modelVersion: gemini-2.5-flash
125+
responseId: bJUUaazPDI-Kn9kPwNOc-AQ
126+
usageMetadata:
127+
candidatesTokenCount: 48
128+
candidatesTokensDetails:
129+
- modality: TEXT
130+
tokenCount: 48
131+
promptTokenCount: 19
132+
promptTokensDetails:
133+
- modality: TEXT
134+
tokenCount: 19
135+
thoughtsTokenCount: 153
136+
totalTokenCount: 220
137+
trafficType: ON_DEMAND
66138
status:
67-
code: 500
68-
message: Internal Server Error
139+
code: 200
140+
message: OK
69141
version: 1

tests/models/test_google.py

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -3121,14 +3121,15 @@ class TreeNode(BaseModel):
31213121

31223122

31233123
async def test_google_recursive_schema_native_output_gemini_2_5(
3124-
allow_model_requests: None, google_provider: GoogleProvider
3125-
):
3126-
"""Test recursive schemas with $ref and $defs using gemini-2.5-flash.
3124+
allow_model_requests: None, vertex_provider: GoogleProvider
3125+
): # pragma: lax no cover
3126+
"""Test recursive schemas with $ref and $defs using gemini-2.5-flash on Vertex AI.
31273127
3128-
NOTE: This test consistently returns a 500 Internal Server Error from Google's API
3129-
as of 2025-11-12. This needs to be investigated and resolved before merging.
3128+
NOTE: Recursive schemas with gemini-2.5-flash FAIL on GLA (500 error) but PASS on Vertex AI.
3129+
This test uses vertex_provider to demonstrate the feature works on Vertex AI.
3130+
The GLA issue needs to be reported to Google.
31303131
"""
3131-
m = GoogleModel('gemini-2.5-flash', provider=google_provider)
3132+
m = GoogleModel('gemini-2.5-flash', provider=vertex_provider)
31323133

31333134
class TreeNode(BaseModel):
31343135
"""A node in a tree structure."""

0 commit comments

Comments
 (0)