[v0.7.8] No option to select from a list of models when using modelSpecs or agents?

[IT-AI-Shen]

Hi,

We were using modelSpecs to customize user experience and 'brand' the conversation (using promptPrefix, greeting etc.) while providing access to several Google Gemini models for the user to choose from.

With v0.7.8 and this PR implemented:#6346 it seems modelSpecs is limited to only one model. There is no model selection sub-menu available in the new endpoints menu.

I looked into implementing something similar through an Agent but there it seems the same limit exists and the agent is fixed to using one model.

Another option was to try and define a custom endpoint but unfortunately this does not work with Google GenAI API.

Why was this option taken away? Why not allow the the choice of using different models while still maintaining the goal of modelSpecs to simplify user experience?

Is there a way to achieve the setup we had before, where we could customize conversations but let users select which model to use, in version 0.7.8?

The only way I can think of is to replicate the modelSpecs list item multiple times for each available model, but this is a messy workaround that would look terrible as a menu.

Appreciate the response,
Itai.

[danny-avila]

Hi @IT-AI-Shen

Features were not taken away, they were refactored into a cleaner solution.

You can see from my comment here that this is quite possible:

#7634 (comment)

[IT-AI-Shen]

Hi @danny-avila and thank you for the quick response!

In the other thread, the concern was about restricting the available options. Our issue is that in versions 0.7.7 and earlier, when a modelSpec preset was chosen, selecting a model from the model menu next to it would retain all the settings from the preset, including the icon, model label, greeting, and promptPrefix. However, this is no longer the case. Even if I allow a Google endpoint, when a user selects it and then a model from it, the preset settings are not applied. So instead of having the chat start with (let's say) “Your friendly AI assistant,” the user is greeted with “Gemini” and the Gemini icon.

I actually have tried testing with "addedEndpoints" to understand its function:

• .env: ENDPOINTS = "openAI,google'
• librechat.yaml: interface.modelSelect set to "true"
• modelSpecs.list[0].addedEndpoints: ["google"]
• enforce true/false (tested both)
• prioritize true/false (tested both)

The expected result would be to see our preset and then only the Google endpoint, but In all cases, both openAI and google endpoints were shown in the list. Either it's a bug or there is another conflicting configuration.

Regardless, as I mentioned above, choosing a model from the Google endpoint underneath our preset, would remove all preset settings.

I genuinely hope I’m overlooking something or that we’re encountering a bug. I would greatly appreciate any further help in resolving this issue.

Thank you,
Itai.

[danny-avila]

You mind sharing your config so I can test your expected behavior? Everything you described should still be the case and is still the case in my tests.

[danny-avila]

Apologize if I misunderstood, today’s been a long travel day. I think I understand your concern. You basically leveraged both model specs and the endpoints + models menu together to create the experience you’re describing. Now that the menu has been consolidated, there seems to be a “regression.” Seeing your config for the older version and noting how it behaves between versions will still help me understand better. I’ll keep this in mind with a potential way to add something like that. It’s definitely not my intention to take away features.

[IT-AI-Shen]

No problem, thank you for taking the time to re-examine this again. I will be back in the office on Tuesday and will share the config. IIRC, under interface we have the endpoints menu and models menu enabled and under modelSpecs.preset we are using endpoint 'google' but not including the 'model' property. Both 'enforce' and 'prioritize' are set to false. This was enough to be able to have the modelSpecs preset applied while our users still being able to select different Gemini models.

[IT-AI-Shen]

@danny-avila
Attached is our production version of librechat.yaml for v0.7.7. The only thing I changed in the file is substituting the original text strings with dummy ones.

librechat.yaml.txt