Suggestion: Gemini-exp support with v1beta?

[mchatzis]

https://github.com/danny-avila/LibreChat/blob/b01c744eb8ba7eb86a120553dd02ff02e85be5c5/api/app/clients/GoogleClient.js#L71C1-L71C5

Are we not accessing the v1beta by design?

I had to change it locally to access the experimental models.

[danny-avila]

Using VertexAI, I can access exp models with latest build

Need more information as to what the change is and what model isn't working otherwise.

[mchatzis]

Oh, I am not using Vertex AI, just got an API key from Google Studio.

Trying to access the experimental models threw an error that they don't exist on v1 API.

I changed it manually to v1beta and they worked.

[danny-avila]

Same thing, using Gemini API (google studio), it works fine with no codebase changes

[danny-avila]

you may need to update to latest depending on what build you're on

[andrey-lepekhin]

I have the same problem even on fresh build @ ubuntu. Here's what I did:

• Cloned the repo
• cp .env.example .env
• Provided GOOGLE_KEY, uncommented GOOGLE_MODELS and added gemini-exp-1206,.
• docker compose up, made the admin account, logged in
• I see the 1206 model in the web UI dropdown, but when I try to send message there:

[handleAbortError] AI response error; aborting request: [GoogleGenerativeAI Error]: Error fetching from https://generativelanguage.googleapis.com/v1/models/gemini-exp-1206:streamGenerateContent?alt=sse: [404 Not Found] models/gemini-exp-1206 is not found for API version v1, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.

This can be fixed by using v1beta of Google API instead of v1 that's being used; would love to know how to make LibreChat use v1beta.

[andrey-lepekhin]

Decided to patch api/app/clients/GoogleClient.js:
Instead of

    return new ChatGoogleGenerativeAI({ ...clientOptions, apiKey: this.apiKey });

Put

    requestOptions = {
      apiVersion: 'v1beta',
    };
    return new ChatGoogleGenerativeAI({ ...clientOptions, apiKey: this.apiKey, requestOptions, });

And then in docker-compose.yaml
From
image: ghcr.io/danny-avila/librechat-dev:latest
To

build:
      context: .
      dockerfile: Dockerfile

...and run docker-compose up --build.

It worked in the end, but while on the way to make docker build without errors I ended up running docker system and volume prunes - so it might be that it actually works on a fresh install as @danny-avila mentions and my docker setup was contaminated with older versions.

[Toss3]

Anyone know how to adjust the output token amount? Seems like it's locked to max 8192 tokens when the actual value should be 65536 (max supported) for gemini-2.0-flash-thinking-exp-01-21.