-
Notifications
You must be signed in to change notification settings - Fork 76
Add Gemma mode #40
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add Gemma mode #40
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
jfschmakeit
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately I'm having trouble running the app with this change:
I'm running into this error when trying to go through the itinerary planning part of the app:
[error] Error[textRefinement > textRefinement > generate > googleai/gemma-3-27b-it, Error] [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1beta/models/gemma-3-27b-it:generateContent: [400 Bad Request] JSON mode is not enabled for models/gemma-3-27b-it
Another question: Do we need to change anything with the embedded vectors now that we are switching to Gemma? I'm wondering if these need to be regenerated or if there's a configuration that needs to be changed?
nohe427
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jfschmakeit - Maybe something is getting past me, but I think we want the output schema defined as that prevents us from needing to parse the genkit responses ourselves.
|
@nohe427 - I think we resolved it in the chat, but yes - it looks like structured output isn't currently supported for Gemma in Genkit (at the moment?). I agree that this is not ideal and potentially a little fragile, but including the response schema in the prompt seems to be working quite well for the current version of Gemma (based on my limited testing). Happy for this to get merged, but let's put it in its own branch ( |
The earliest version of Genkit that has the Gemma models in it is 1.14.1.
Seems that their implementation doesn't support JSON mode yet.
So there's slight parsing that needed to be done.
Seems like other folks were having the same problem:
https://discuss.ai.google.dev/t/gemma-3-missing-features-despite-announcement/71692/14