Skip to content

Google GenAI SafeSettings change in 1.14.4 breaks Gemini API usage #2146

@andersio

Description

@andersio
  • This is actually a bug report.
  • I am not getting good LLM Results
  • I have tried asking for help in the community on discord or discussions and have not received a response.
  • I have tried searching the documentation and have not found an answer.

What Model are you using?

gemini-3-flash

Describe the bug
The SafetySettings changes in https://github.com/567-labs/instructor/pull/2007/changes are incompatible with the Gemini API.

The Gemini API does not support these IMAGE_ harm categories as per https://ai.google.dev/api/generate-content#harmcategory.

The IMAGE_ family of harm categories are only available through Vertex AI API.
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/reference/rest/v1/HarmCategory

To Reproduce
Run Instructor 1.14.4-1.14.5 with google/gemini-3-flash-preview, and attach any kind of image.

Observe Instructor failing with:

    400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'Invalid value at \'safety_settings[0].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_HATE"\nInvalid value at \'safety_settings[1].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_DANGEROUS_CONTENT"\nInvalid value at \'safety_settings[2].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_HARASSMENT"\nInvalid value at \'safety_settings[3].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_SEXUALLY_EXPLICIT"', 'status': 'INVALID_ARGUMENT', 'details': [{'@type': 'type.googleapis.com/google.rpc.BadRequest', 'fieldViolations': [{'field': 'safety_settings[0].category', 'description': 'Invalid value at \'safety_settings[0].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_HATE"'}, {'field': 'safety_settings[1].category', 'description': 'Invalid value at \'safety_settings[1].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_DANGEROUS_CONTENT"'}, {'field': 'safety_settings[2].category', 'description': 'Invalid value at \'safety_settings[2].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_HARASSMENT"'}, {'field': 'safety_settings[3].category', 'description': 'Invalid value at \'safety_settings[3].category\' (type.googleapis.com/google.ai.generativelanguage.v1beta.HarmCategory), "HARM_CATEGORY_IMAGE_SEXUALLY_EXPLICIT"'}]}]}}

Expected behavior
Existing Gemini image extraction routines work on Instructor <1.14.4, using google.genai client, Gemini API and Gemini API Key.

It breaks on 1.14.4 and 1.14.5.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:criticalCritical issue affecting productiontype:regressionRegression from previous version

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions