Seeking Assistance with Azure OpenAI GPT-4o Configuration

[mraguth]

I understand you may see multiple comments from me. Has anyone had success using 'Azure OpenAI GPT-4o'? If so, could you please share your experience? My current configuration isn't working when I switch to GPT-4o. Any help would be greatly appreciated.

Working config:

llm = AzureOpenAI(
    model="gpt-4",
    deployment_name="gpt-4",
    api_key=api_key,
    azure_endpoint=azure_endpoint,
    api_version=api_version,
    temperature=0,
)

Non working config:

llm = AzureOpenAI(
    model="gpt-4o",
    deployment_name="gpt-4o",
    api_key=api_key,
    azure_endpoint=azure_endpoint,
    api_version=api_version,
    temperature=0,
)

[mraguth]

This is issue with utilis.py (the file in the webiste has up to date with all models. However, my pc file is not updated despite I upgrade the package. I replace with below and error has been resolved. Closing the ticket.

llama_index/llama-index-integrations/llms/llama-index-llms-openai/llama_index/llms/openai/utils.py

Line 59 in 5cd850f

AZURE_TURBO_MODELS: Dict[str, int] = {

[dosubot[bot]]

To configure the AzureOpenAI class to work correctly with the GPT-4o model, ensure you have the following settings:

1. Environment Variables:

   • OPENAI_API_VERSION: Set this to 2023-07-01-preview or newer.
   • AZURE_OPENAI_ENDPOINT: Your endpoint should look like https://YOUR_RESOURCE_NAME.openai.azure.com/
   • AZURE_OPENAI_API_KEY: Your API key if the API type is azure.

2. Configuration Parameters:

   • Ensure you specify the engine parameter to identify your deployment.

Here is an example configuration:

from llama_index.llms.azure_openai import AzureOpenAI

aoai_api_key = "YOUR_AZURE_OPENAI_API_KEY"
aoai_endpoint = "YOUR_AZURE_OPENAI_ENDPOINT"
aoai_api_version = "2023-07-01-preview"

llm = AzureOpenAI(
    model="gpt-4o",
    deployment_name="gpt-4o",
    api_key=aoai_api_key,
    azure_endpoint=aoai_endpoint,
    api_version=aoai_api_version,
    temperature=0,
)

Additionally, ensure your utils.py file is up to date. The specific change that might be necessary is:

enc = tiktoken.encoding_for_model("gpt-3.5-turbo")

This line sets the tokenizer to use the encoding for the "gpt-3.5-turbo" model, which likely resolves compatibility issues with the GPT-4o model ^[1].

If you continue to face issues, make sure all local files are up to date with the latest package updates from the LlamaIndex repository by using the upgrade command:

llama-index-cli upgrade <directory>

Or for a single file:

llama-index-cli upgrade-file <path>

These steps should help resolve the configuration issues with the GPT-4o model.

To continue talking to Dosu, mention @dosu.