Does gemini cli support connection to gemini models on SAP AI Core ?

[AjitKP91]

SAP AI Core (Gen AI Hub) provide Gemini models on their platform. There are other platforms as well. Is there a way to consume gemini models from these platforms? is it possible to provide SAP AI Core credentials and then use gemini cli?

[acoliver]

https://help.sap.com/docs/ai-launchpad/sap-ai-launchpad/models-and-scenarios-in-generative-ai-hub-fef463b24bff4f44a33e98bb1e4f3148 - sap seems to provider an openai endpoint.

Gemini-cli does not right now.

https://github.com/acoliver/llxprt-code
npm install -g @vybestack/llxprt-code

is a downstream fork that keeps up with the releases of gemini-cli.

You could do

llxprt --baseurl $YOURSAPBASEURL --key $YOURKEY --provider openai
/model would give you a menu or you could supply --model at the command line.

then you could use this downstream fork of gemini-cli with whatever model you like on the SAP hub.

[acoliver]

Oh badass. I'll have to try that!

[Manamama]

FYI, real badass is that: https://github.com/Piebald-AI/gemini-desktop - it works in Ubuntu but failed at the last step in prooted Debian in Termux (and earlier in Termux itself), see: https://github.com/Piebald-AI/gemini-desktop with my informal attempts.

Another fun ouroboros: https://github.com/Manamama-Gemini-Cloud-AI-01/customizations-of-gemini-cli/blob/main/how_to_modify_system_message.md, where I had Gemini Cloud AI extract the semi-secret method to modify the silly default system.md, which tends to make her (Gemini Cloud AI when in Gemini CLI) this: #4556 (comment) too often.

But I digress, sorry...

[acoliver]

Llxprt system messages as of 0.1.17 or so are completely customizable. In ~/.llxprt/prompts you can find the defautls and you can override them for all or by provider and model. Even per tool. Piebald looks like theyre really working hard. I'd love to invite them downstream.

We can get those hooks into llxprt whether gemini-cli wants them or not. Many look like obvious improvements or extension points. And gemini desktop well that's nice but WhateverTheHeckModel Desktop seems cooler to me at least!

[acoliver]

Oh I can probably cherrypick his stuff too.. and please send pull requests for the default prompts. I'd love to make them shorter. I think some of the repetition will be fixed because I murdered nextSpeakerChecker but... anyhow in llxprt they are in md files that get installed in your home config. Once you test an optimization with a couple models please pr me because optimization is on my list....they are soooo long.

[acoliver]

🙏

[AjitKP91]

@acoliver
SAP uses oAuth for authentication instead of apikey.
SAP Cloud SDK for AI

Is there a way to support oauth?

[acoliver]

Yes. In fact I've been wanting to find at least one other oauth provider to help shore up the architecture. I'll look into this tomorrow.

As long as I can get access to test with it shouldn't be too hard with their sdk and such. Connect with me on LinkedIn or wherever. I'll ping you for testing.

Shouldn't be hard. The thing about SAP is their engineering is usually solid.

[acoliver]

Note I just added Qwen's oauth. The trouble with SAP is it takes more time/trouble to get just access to their provider than it did to implement this for qwen

[AjitKP91]

Okay. Can you share the pr for qwen?

[acoliver]

acoliver@3030bf8

acoliver@1b2302a