Issues with Local Deployment connecting to AOAI

[timothymeyers]

I'm trying to work through the Local Deployment as described in the LocalDeployment.md file.

Everything is deployed per the instructions with both Cosmos and AOAI resources running in Azure and my .env file is set up per the instructions.

I can run the backend and frontend per the instructions. Backend /healthz returns OK and Frontend loads on :3000.

Whenever I attempt to generate a Plan, the logs show a 404 for the AOAI resource access. The chat is stored to the Cosmos memory, however, so that is working.

It looks like it is having challenges authenticating to the AOAI instance. I've tried to debug and noticed within auth_utils.py that the user_object is being populated with dummy data rather than using my DefaultAzureCredential, though I'm not sure exactly what the intention is here.

I'm a bit unsure how to proceed to get the backend to properly auth with AOAI when running locally.

[Vinay-Microsoft]

Hi @timothymeyers will forward this Issue to our engineering team and keep you posted.

[Prashant-Microsoft]

Hi @timothymeyers, Thank you for reporting this issue! We appreciate your feedback, and we will start investigating the issue and keep you updated on our progress.

[Prashant-Microsoft]

Hi @timothymeyers, To assist you further, could you please provide the following details:

1. Which Bicep file are you using for local deployment? Is it macae-dev.bicep?
2. Could you provide more information about which API is failing? A screenshot of the error would be very helpful.
3. For local purposes, we are not using the auth/me API; instead, we are working with sample data.
4. Please share any error details, including relevant screenshots, to help us better understand the issue.

Your assistance in providing this information will allow us to investigate the issue more effectively.

[Prashant-Microsoft]

Hi @timothymeyers,
We just wanted to follow up on our previous message regarding the issue you're facing. We still need some additional details to assist you further. When you have a moment, could you please provide the requested information? It will help us investigate the problem more effectively.

[timothymeyers]

Hi @Prashant-Microsoft - thanks; I'm working on reproducing the issue to get you some screenshots. Right now, I'm just getting 429 errors when accessing the AOAI models. Before, I was getting a 404 and when I inspected the request headers they had dummy user profile data in them.

1. Yes I'm using macae-dev.bicep

Standby.

[timothymeyers]

Update: I've been able to get the local deployment to work. The 429 errors were the clue.

TL;DR - the accelerator makes an assumption that the AOAI gpt-4o Deployment and Model names are going to be the same, because that is how the bicep file deploys them. It does not work when using an existing AOAI Resource and gpt-4o Deployment where the Deployment Name and Model Name are different.

• The 429 errors were occurring while using the AOAI deployment created by the macae-dev.bicep file. I already have a few instances of AOAI and model deployments, so I was capped out and the gpt-4o deployment done here would just not work.
• To get around this before and this time, I updated my .env to point at one of my existing AOAI and gpt-4o model deployments. This in turn resulted in 404 errors.
• The 404 was because my Deployment name was gpt4o and my model name was gpt-4o.
• I updated src/backend/config.py to be able to handle Model Name separately, and everything appears to be working now. Diff below.

@Prashant-Microsoft would you want me to submit this as a PR?

diff --git a/src/backend/config.py b/src/backend/config.py
index bf12609..8fb6718 100644
--- a/src/backend/config.py
+++ b/src/backend/config.py
@@ -36,6 +36,7 @@ class Config:
     COSMOSDB_CONTAINER = GetRequiredConfig("COSMOSDB_CONTAINER")
 
     AZURE_OPENAI_DEPLOYMENT_NAME = GetRequiredConfig("AZURE_OPENAI_DEPLOYMENT_NAME")
+    AZURE_OPENAI_MODEL_NAME = GetRequiredConfig("AZURE_OPENAI_MODEL_NAME")
     AZURE_OPENAI_API_VERSION = GetRequiredConfig("AZURE_OPENAI_API_VERSION")
     AZURE_OPENAI_ENDPOINT = GetRequiredConfig("AZURE_OPENAI_ENDPOINT")
     AZURE_OPENAI_API_KEY = GetOptionalConfig("AZURE_OPENAI_API_KEY")
@@ -81,13 +82,16 @@ class Config:
         return get_bearer_token_provider(Config.GetAzureCredentials(), scopes)
 
     def GetAzureOpenAIChatCompletionClient(model_capabilities):
         if Config.__aoai_chatCompletionClient is not None:
             return Config.__aoai_chatCompletionClient
 
         if Config.AZURE_OPENAI_API_KEY == "":
             # Use DefaultAzureCredential for auth
             Config.__aoai_chatCompletionClient = AzureOpenAIChatCompletionClient(
-                model=Config.AZURE_OPENAI_DEPLOYMENT_NAME,
+                model=Config.AZURE_OPENAI_MODEL_NAME,
+                azure_deployment=Config.AZURE_OPENAI_DEPLOYMENT_NAME,
                 api_version=Config.AZURE_OPENAI_API_VERSION,
                 azure_endpoint=Config.AZURE_OPENAI_ENDPOINT,
                 azure_ad_token_provider=Config.GetTokenProvider(
@@ -97,14 +101,18 @@ class Config:
                 temperature=0,
             )
         else:
             # Fallback behavior to use API key
             Config.__aoai_chatCompletionClient = AzureOpenAIChatCompletionClient(
-                model=Config.AZURE_OPENAI_DEPLOYMENT_NAME,
+                model=Config.AZURE_OPENAI_MODEL_NAME,
+                azure_deployment=Config.AZURE_OPENAI_DEPLOYMENT_NAME,
                 api_version=Config.AZURE_OPENAI_API_VERSION,
                 azure_endpoint=Config.AZURE_OPENAI_ENDPOINT,
                 api_key=Config.AZURE_OPENAI_API_KEY,
                 model_capabilities=model_capabilities,
                 temperature=0,
             )
 
         return Config.__aoai_chatCompletionClient

[Vinay-Microsoft]

@timothymeyers Please submit the PR and our team will review it

[Roopan-Microsoft]

Hey @timothymeyers - Thanks for the PR. We are looking into it.

Also, we are looking if there are any other changes required like bicep, .env file. We will validate once and merge it.

Thanks again for the feedback.

[Prashant-Microsoft]

Hi @timothymeyers,
Thank you for your patience. we have addressed the issue you raised and also implemented the necessary end-to-end changes. please pull the latest code and let us know if you have any further questions or feedback.

Thanks again for your contributions!

[Roopan-Microsoft]

Thanks @timothymeyers, for your contribution. We have fixed as part this PR - #59.

Please let us know if any. Always happy to help!

We are closing this issue for now. Thanks.