Keep getting Deployment not found when switching to Azure OpenAI from OpenAI

[manojahi]

Hi,

I have been using OpenAI with langchain in my application and recently tries to switch to Azure OpenAI but keep getting error - Error: Azure OpenAI API deployment name not found.

Following is the code I am using

import { AzureChatOpenAI } from '@langchain/openai';

  const llm = new AzureChatOpenAI({
    temperature: 0,
    azureOpenAIApiDeploymentName: 'GPT35turbo16k',
    azureOpenAIApiVersion: '0613',
    azureOpenAIApiInstanceName: 'https://my-instance-name.openai.azure.com/',
    azureOpenAIApiKey: 'my-api-key',
    streaming: true,
    verbose: true,
    callbacks: [
      {
        handleLLMNewToken(token: string) {
          res.write(
            JSON.stringify({
              type: 'html',
              content: token,
            }),
          );
        },
      },
    ],
  });

I already created deployment in Azure and using deployment name instead of model name. screenshot of azure attached.

Edit:

I am using following version of langchain
"langchain": "^0.1.19",

any help is appreciated. Thanks

[leitdeux]

It seems that with Langchain v0.2, constructing AzureChatOpenAI has changed-- once I updated from v0.1 my use of the AzureChatOpenAI constructor also broke like yours, and last time I checked, the documentation wasn't clear on what parameters were needed in v0.2 onward.

I solved it by doing two things:

1.) in my .env file, I set the following environmental vars (this is for use with "gpt-4o" specifically, your api version may differ if using a different model):

OPENAI_API_TYPE=azure
AZURE_OPENAI_API_VERSION=2024-05-01-preview
AZURE_OPENAI_ENDPOINT=https://MY_AZURE_OPENAI_RESOURCE_NAME.openai.azure.com
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_API_DEPLOYMENT_NAME=MY_OPENAI_DEPLOYMENT_NAME # e.g. gpt-4o
AZURE_OPENAI_API_INSTANCE_NAME=MY_OPENAI_MODEL_NAME_HERE  # e.g. gpt-4o

2.) In the initialization code of AzureChatOpenAI, passing in the token provider azureADTokenProvider, and azureOpenAIBasePath:

import dotenv from 'dotenv';
dotenv.config();

import {
  DefaultAzureCredential,
  getBearerTokenProvider
} from '@azure/identity';

function getAzureBearerTokenProvider() {
  return getBearerTokenProvider(
    new DefaultAzureCredential(),
    'https://cognitiveservices.azure.com/.default'
  );
}

const azureADTokenProvider = await getAzureBearerTokenProvider();

const llm = new AzureChatOpenAI({
  azureADTokenProvider,
  azureOpenAIBasePath: `${process.env.AZURE_OPENAI_ENDPOINT}/openai/deployments`,
  model: process.env.AZURE_OPENAI_API_INSTANCE_NAME,
  temperature: 0,
  streaming: true
}));

[manojahi]

Thanks for your response @leitdeux

I checked that I am using "langchain": "^0.1.19",, Currently not planning to upgrade langchain to v0.2 to avoid breaking of other things as I just want to start using Azure OpenAI instead of OpenAI Directly.

Do you have any reference of your usage of Azure OpenAI when you were using v0.1?

Thanks again

[leitdeux]

Hi @manojahi, in the case of v0.1 Langchain, I believe I just followed the documentation as-is regarding the AzureChatOpenAI configuration object needed to construct it. But I wonder if perhaps it might be your version of @langchain/openai that might be why you're suddenly getting errors now? Perhaps there might be a version mismatch between that package and Langchain v0.1 in your project? Just an idea.

[manojahi]

I just checked, following are the versions I am using

    "@langchain/azure-openai": "^0.0.10",
    "@langchain/openai": "^0.0.14",
    "@langchain/pinecone": "^0.0.3",

let me check if openai also have v0.1