403 Forbidden when ai search and azure openai public networking is disabled

[GijsVoogd]

Question regarding Azure OpenAI using RAG on our own data via AI Search.

Our setup is as follows:

AI Search:

• private network disabled
• private endpoint for inbound traffic
• shared private link for openai (type account), and blob storage (type blob and dfs)
• allow Azure services on the trusted services list to access this cognitive services account

Azure OpenAI:

• private network disabled
• private endpoint for inbound traffic (the openai account and search shared private link)
• allow Azure services on the trusted services list to access this cognitive services account

The issue we’re facing is that it is calling Azure OpenAI and from Azure OpenAI, AI Search is called. This gives a 403 which seems to make sense since AI Search public network is disabled and we cannot integrate Azure OpenAI in the VNET. Shared private access is not an option on Azure OpenAI.

It works when we enable public access on AI Search but we have a requirement for private access only. Is there a way to make this work without enabling public access?

{

"error": {

"requestid": "xxx",

"code": 400,

"message": "Invalid AzureCognitiveSearch configuration detected: Call to get Azure Search index failed. Check if you are using the correct Azure Search endpoint and index name. If you are using key based authentication, check if the admin key is correct. If you are using access token authentication or managed identity of Azure OpenAI, check if the Azure Search has enabled RBAC based authentication and if the user identity or Azure OpenAI managed identity has required role assignments to access Azure Search resource
[https://aka.ms/aoaioydauthentication].
If the Azure Search resource has no public network access, make sure enable trusted service of Azure Search.\nAzure Search Error: 403, message\u003d\u0027Server responded with status 403.

[pamelafox]

Hm, is this an issue you're encountering with this repository, or when using the Azure OpenAI on your Data feature?

[GijsVoogd]

@pamelafox I am using this repo but this issue, i think, is not related to it. Looks like it is more related to azure components. Having this would be a great addition to the productionizing part of the repo

[amddaa]

@GijsVoogd Any update on that? I'm facing similar issue using api_key to authenticate (system_assigned_managed_identity works fine).

[GijsVoogd]

@amddaa I also switched to MI authentication which is the preferred way anyways. Never got the key authentication working

[sebastus]

I have this exact issue in a python program. I'm not using this repo. I'm using the AzureOpenAI import from pypi openai.

chat_completion = openai_client.chat.completions.create(
    model=deployment,
    messages=chat_history,
    max_tokens=800,
    temperature=0.7,
    top_p=0.95,
    frequency_penalty=0,
    presence_penalty=0,
    stop=None,
    stream=False,
    extra_body={
        "data_sources": [
            {
                "type": "azure_search",
                "parameters": {
                    "endpoint": f"{azure_search_endpoint}",
                    "index_name": azure_search_index_name,
                    "semantic_configuration": "default",
                    "query_type": "simple",
                    "fields_mapping": {},
                    "in_scope": True,
                    "role_information": "some stuff",
                    "filter": None,
                    "strictness": 3,
                    "top_n_documents": 5,
                    "authentication": {
                        "type": "api_key",
                        "key": f"{azure_search_admin_key}"
                    }
                }                
            }
        ]
    }
)

I delete my environment nightly and rebuild it in the morning. Yesterday, it all worked correctly. But all other days (about 2 weeks worth?) this fails. I have private endpoints on both openai and ai search. Public access is enabled for openai at the moment, but public access is disabled for ai search.

[arnabbiswas1]

Apart from the configurations mentioned in the original question, following is also needed:

• Managed identity between Azure Open AI and Azure AI Search. Key based authentication didn't work. We used user assigned managed identify

• RBAC roles mentioned in the following wiki. For me the Azure Open AI on your data was hosted on a flask server which in turn was running on a k8s cluster. So following roles were sufficient: Search Index Data Reader, Search Service Contributor, Cognitive Services OpenAI Contributor

• For the private endpoint, the regular Open AI endpoint was configured against the private IP for DNS resolution

Bottom line, follow the wiki: https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/use-your-data-securely#role-assignments

[alexandreweiss]

@pamelafox Same here that Arna : seems the issue is with OpenAI communicating with OpenAI search not using private endpoint. Thus the complete private chain cannot be used especially is the python app accessing openAI is not part of the vnet but rather part of a routed private network (like seating in AWS)

[GijsVoogd]

Swapping the key for the managed identity is the solution here

[alexandreweiss]

Thanks @GijsVoogd what do you mean by swapping keys? Keys from the Openai search? Or keys on the managed identity in EntraID from Openai and Openai search?

[GijsVoogd]

See if this is an option for you: Azure/azure-sdk-for-net#45326

[alexandreweiss]

Sorry but cannot use MI as my app front-end runs in AwS and uses openai and search in an Azure subscription. I have private connectivity and DNS to resolve Private endpoint between Azure and AWS. Search already has 'both' enabled for authn. It seems app is reaching out to Openai that it turn reaches out to search. In the last part, something might be wrong on how Openai calls search. Perhaps not using MI...

[alexandreweiss]

Hi @pamelafox : Finally found my problem. Seems the webapp (that I run on AWS side) talks to OpenAI (using API Key)
In turn, Open AI talks to AI Search. To force that second traffic to use MI, you must NOT provide API KEY for AI Search.
So only providing API Key for Open AI to the WebApp along with only the name of the AI search service + index name made it for me !

[pamelafox]

This repo now has support for private endpoints and deployment for both Container Apps and App Service, so I am closing this issue. Please open a new one if using the private endpoint feature and encountering issues.