Is it posible to run it via API Management service?

[perssonkasia]

Hello,
I would like to protect my Azure Open AI API's with Azure API Management service.
Unfortunately, when i change AZURE_OPENAI_ENDPOINT to my APIM url and AZURE_OPENAI_KEY to subscription key, chat doesn't work anymore.
I will appreciate for any help.

[npsquared]

Hi @perssonkasia, this repo maybe useful to you: https://github.com/Azure-Samples/openai-python-enterprise-logging

[perssonkasia]

Hi @npsquared , thank you for the link.
My APIM settings works fine. I can call Azure Open AI API via APIM API from Postman, I get the response.
But when I switch in azure app service Configuration setting AZURE_OPENAI_ENDPOINT to https://apim.XXXXXXXX.azure-api.net/ and AZURE_OPENAI_KEY to APIM subscription key, chat not working anymore. I receive the error:

Error
There was an error generating a response. Chat history can't be saved at this time. Error code: 404 - {'statusCode': 404, 'message': 'Resource not found'}

[p-n-rao]

We are also looking for this (planning to add APIM between APP and openAI).

[borrvick]

Just altering the init_openai_client() method and adding APIM subscription key to the header did the trick for me.

`
def init_openai_client(use_data=SHOULD_USE_DATA):
azure_openai_client = None
try:
# API version check
if (
AZURE_OPENAI_PREVIEW_API_VERSION
< MINIMUM_SUPPORTED_AZURE_OPENAI_PREVIEW_API_VERSION
):
raise Exception(
f"The minimum supported Azure OpenAI preview API version is '{MINIMUM_SUPPORTED_AZURE_OPENAI_PREVIEW_API_VERSION}'"
)

           # Endpoint
    if not APIM_ENDPOINT:
        raise Exception(
            "APIM_ENDPOINT is required"
        )

    endpoint = (
        APIM_ENDPOINT
    )

    # Authentication
    aoai_api_key = AZURE_OPENAI_KEY
    ad_token_provider = None
    if not aoai_api_key:
        logging.debug("No AZURE_OPENAI_KEY found, using Azure AD auth")
        ad_token_provider = get_bearer_token_provider(
            DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"
        )


    # Deployment
    deployment = AZURE_OPENAI_MODEL
    if not deployment:
        raise Exception("AZURE_OPENAI_MODEL is required")

    # Default Headers
    default_headers = {"x-ms-useragent": USER_AGENT,
            'Ocp-Apim-Subscription-Key': APIM_SUBSCRIPTION_KEY
            }

    azure_openai_client = AsyncAzureOpenAI(
        api_version=AZURE_OPENAI_PREVIEW_API_VERSION,
        api_key=aoai_api_key,
        azure_ad_token_provider=ad_token_provider,
        default_headers=default_headers,
        azure_endpoint=endpoint,
    )

    return azure_openai_client
except Exception as e:
    logging.exception("Exception in Azure OpenAI initialization", e)
    azure_openai_client = None
    raise e

`