Add option for custom API path

[rostrovsky]

Problem statement

Currently OpenAIClient relies on hardcoded f"{self.api_host}/v1/chat/completions" API path.

Microsoft Azure allows private deployments of OpenAI GPT models: https://azure.microsoft.com/en-us/pricing/details/cognitive-services/openai-service/

These models have a completely different API path which looks like: https://<something>.openai.azure.com/openai/deployments/GPT35/chat/completions (this is real world example).

As of now it's impossible to use shell_gpt with models deployed that way.

Proposed solution

Introduce OPENAI_API_PREFIX configuration variable that will allow replacing v1 with user-defined API path.

I will happily introduce this change as long as it will gets your approval. Thanks!

Benefit

• Unblocks usage of shell_gpt in restricted/enterprise environments.

[bennettnw2]

Came here as I just got blocked from using OpenAI in my enterprise environment. Please, please, please do this!

@rostrovsky, from the "Contributing to ShellGPT" section I see this:

If you have a new feature in mind that doesn't have an existing issue, kindly create a discussion in the "ideas" category using GitHub Discussions. Gather feedback from the community, and if you receive approval from at least a couple of people, create an issue and assign it to yourself.

So you have one approval from me!

[bennettnw2]

Upon looking into this further, does this PR solve the above problem?
#307

[rbgnr111]

I got it working in our with the azure aopenai instance we use.
I had to make the following changes to ~/.local/lib/python3.10/site-packages/sgpt/client.py

#model is not supported in azure openai and throws an error
orig: data = {
"messages": messages,
"model": model,
"temperature": temperature,
"top_p": top_probability,
"stream": stream,
}
change: data = {
"messages": messages,
#"model": model,
"temperature": temperature,
"top_p": top_probability,
"stream": stream,
}

#like mentioned earlier, change the path mirosoft seems to reference the model in the path
orig: endpoint = f"{self.api_host}/v1/chat/completions"
change: endpoint = f"{self.api_host}/GPTAPI/GPT35Turbo"

#authorization is looking for "SubscriptionID" header rather than a bearer token
orig: "Authorization": f"Bearer {self.__api_key}",
change: "SubscriptionID": f"{self.__api_key}",

#the format returned is a little different I get "message" rather than "delta"
orig: delta = data["choices"][0]["delta"] # type: ignore
change delta = data["choices"][0]["message"] # type: ignore

[mwester117]

This issue should get fixed with this PR: #206
Upvote to integrate this in the next release

[TheR1D]

@rostrovsky In the latest releases new OPENAI_BASE_URL variable has been added in to config file which allows to define full URL (except /chat/completion). For instance OPENAI_BASE_URL=https://api.openai.com/v1.

[mwester117]

@rostrovsky In the latest releases new OPENAI_BASE_URL variable has been added in to config file which allows to define full URL (except /chat/completion). For instance OPENAI_BASE_URL=https://api.openai.com/v1.

Since you moved to the openai python module the workaround in the PR does not work anymore.
Your suggestion to use the OPENAI_BASE_URL is not working for Azure OpenAI since you need to use the seperated class "AzureOpenAI" instead of "OpenAI".

See also https://learn.microsoft.com/en-us/azure/ai-services/openai/quickstart?tabs=bash%2Cpython-new&pivots=programming-language-python

[mads-oestergaard]

Also, functions cannot be None in the message when using AzureOpenAI.

[TheR1D]

I'm considering to use LiteLLM for multiple backends integration. This should provide an option to switch between various LLM backends including Azure. I would greatly appreciate any assistance with testing the Azure Integration PR #463. To test it follow these steps:

git clone https://github.com/TheR1D/shell_gpt.git
cd shell_gpt
git checkout ollama
python -m venv venv
source venv/bin/activate
pip install -e  .

export AZURE_API_KEY=YOUR_KEY
export AZURE_API_BASE=YOUR_API_BASE
export AZURE_API_VERSION=YOUR_API_VERSION
export AZURE_AD_TOKEN=YOUR_TOKEN # Optional
export AZURE_API_TYPE=YOUR_API_TYPE # Optional

sgpt --model azure/<your_deployment_name> --no-functions "Hi Azure"

[ishaan-jaff]

Thanks for considering LiteLLM @TheR1D ! how can we improve for you ?

[TheR1D]

Thank you for the great project @ishaan-jaff. I don't have any suggestions now, but I will mention you here once I have them.