-
Notifications
You must be signed in to change notification settings - Fork 736
Open
Labels
priority: p2Moderately-important priority. Fix may not be included in next release.Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.Error or flaw in code with unintended results or allowing sub-optimal usage patterns.
Description
Description
When using structured JSON output with response_json_schema, the required constraint in the schema is not enforced. The API accepts and returns responses that are missing fields marked as required without raising an error.
Environment
- SDK:
google-genai - Backend: Vertex AI
- Model:
gemini-3-flash-preview - Python version: 3.11+
Minimal Reproducible Example
from google import genai
from google.genai import types
client = genai.Client(vertexai=True, project="your-project", location="us-central1")
# Schema requires "prompt" field
schema = {
"type": "object",
"properties": {
"prompts": {
"type": "array",
"items": {
"type": "object",
"properties": {
"prompt": {"type": "string"},
"content": {"type": "string"},
"confidence": {"type": "number"},
},
"required": ["prompt"], # <-- "prompt" is REQUIRED
},
}
},
"required": ["prompts"],
}
# System prompt example shows "content" field (not "prompt")
user_prompt = '''
Generate a prompt in this JSON format:
{
"prompts": [
{
"content": "Your generated text here",
"confidence": 0.9
}
]
}
'''
response = client.models.generate_content(
model="gemini-3-flash-preview",
contents=user_prompt,
config=types.GenerateContentConfig(
response_mime_type="application/json",
response_schema=schema,
)
)
print(response.text)
# Returns: {"prompts": [{"content": "...", "confidence": 0.9}]}
# Note: "prompt" field is MISSING but no error is raised!Expected Behavior
One of the following:
- The API should enforce the schema and ensure required fields are present
- OR raise a validation error when the response doesn't include required fields
- OR document that required is advisory only, not strictly enforced
Actual Behavior
The API silently accepts responses missing required fields.
Impact
In production, Gemini sometimes returns responses with only content while our schema requires prompt. Since no error is raised, invalid responses propagate through the system causing hard-to-diagnose bugs.
Additional Context
- Behavior is intermittent - sometimes responses include all fields, sometimes they don't
- The model appears to follow natural language instructions over schema constraints
- When schema says required: ["prompt"] but the prompt example shows "content", the model returns content only
Metadata
Metadata
Assignees
Labels
priority: p2Moderately-important priority. Fix may not be included in next release.Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.Error or flaw in code with unintended results or allowing sub-optimal usage patterns.