Skip to content

Conversation

ryanwilson
Copy link
Member

Adds an optional responseModalities property to the GenerationConfig object in the Vertex AI SDK. This allows specifying the desired modalities (TEXT, IMAGE, AUDIO) for the response.

Includes updates to the GenerationConfig.Builder and internal serialization logic.

Adds unit tests to verify:

  • response_modalities field is omitted when null.
  • Correct serialization for all modalities ([TEXT, IMAGE, AUDIO]).
  • Correct serialization for a subset of modalities ([TEXT, IMAGE]).

Adds an optional `responseModalities` property to the `GenerationConfig`
object in the Vertex AI SDK. This allows specifying the desired modalities
(TEXT, IMAGE, AUDIO) for the response.

Includes updates to the `GenerationConfig.Builder` and internal
serialization logic.

Adds unit tests to verify:
- `response_modalities` field is omitted when null.
- Correct serialization for all modalities ([TEXT, IMAGE, AUDIO]).
- Correct serialization for a subset of modalities ([TEXT, IMAGE]).
Copy link
Contributor

github-actions bot commented Apr 16, 2025

📝 PRs merging into main branch

Our main branch should always be in a releasable state. If you are working on a larger change, or if you don't want this change to see the light of the day just yet, consider using a feature branch first, and only merge into the main branch when the code complete and ready to be released.

Copy link
Contributor

Vertex AI Mock Responses Check ⚠️

A newer major version of the mock responses for Vertex AI unit tests is available. update_responses.sh should be updated to clone the latest version of the responses: v10.0

@google-oss-bot
Copy link
Contributor

1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

@google-oss-bot
Copy link
Contributor

Copy link
Contributor

Test Results

0 files   -    66  0 suites   - 66   0s ⏱️ - 1m 16s
0 tests  -   552  0 ✅  -   551  0 💤  - 1  0 ❌ ±0 
0 runs   - 1 104  0 ✅  - 1 102  0 💤  - 2  0 ❌ ±0 

Results for commit 0e6ba3e. ± Comparison against base commit 51b4a1c.

@google-oss-bot
Copy link
Contributor

The public api surface has changed for the subproject firebase-vertexai:
error: Added field com.google.firebase.vertexai.type.GenerationConfig.Builder.responseModalities [AddedField]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

@ryanwilson ryanwilson closed this Apr 17, 2025
@firebase firebase locked and limited conversation to collaborators May 18, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants