You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can import AI model endpoints deployed in Azure AI Foundry to your API Management instance as a REST API. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
18
18
19
19
Learn more about managing AI APIs in API Management:
20
20
21
-
*[Generative AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
21
+
*[AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
22
22
23
23
24
-
## AI service options
25
-
***Azure OpenAI service** - Deployment name of a model is passed in the URL path of the API request.
24
+
## Client compatibility options
26
25
27
-
***Azure AI** - These are models that are available in Azure AI Foundry through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api). Deployment name of a model is passed in the request body of the API request.
26
+
API Management supports two client compatibility options for AI APIs. The option you select determines how clients call the API and how the API Management instance routes requests to the AI service.
28
27
28
+
***Azure AI** - Manage model endpoints in Azure AI Foundry that are exposed through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api).
29
+
30
+
Clients call the deployment at a `/models` endpoint such as `/Deepseek-3/models/chat/completions`. Deployment name is passed in the request body. Use this option if your service includes models exposed through the Azure AI Model Inference API.
31
+
32
+
***Azure OpenAI Service** - Manage model endpoints deployed in Azure OpenAI Service.
33
+
34
+
Clients call the deployment at an `/openai` endpoint such as `/openai/deployments/my-deployment/chat/completions`. Deployment name is passed in the request path. Use this option if your service only includes Azure OpenAI Service model deployments.
29
35
30
36
## Prerequisites
31
37
32
38
- An existing API Management instance. [Create one if you haven't already](get-started-create-service-instance.md).
33
39
- One or more Azure AI services with models deployed, such as:
34
-
- An Azure OpenAI resource. For information about model deployment in Azure OpenAI service, see the [resource deployment guide](/azure/ai-services/openai/how-to/create-resource).
35
-
- An Azure AI Foundry project. For information about creating a project, see [Create a project in the Azure AI Foundry portal](/azure/ai-foundry/how-to/create-projects).
36
-
37
40
41
+
- An Azure AI service in your subscription with one or more models deployed. Examples include models deployed in Azure AI Foundry or Azure OpenAI Service.
38
42
39
-
## Import AI Foundry API using the portal
43
+
## Import AI API using the portal
40
44
41
-
Use the following steps to import an AI Foundry API directly to API Management.
45
+
Use the following steps to import an AI API to API Management.
@@ -58,19 +62,21 @@ To import an AI Foundry API to API Management:
58
62
59
63
:::image type="content" source="media/azure-ai-foundry-api/ai-foundry-api.png" alt-text="Screenshot of creating an OpenAI-compatible API in the portal." :::
60
64
1. On the **Select AI service** tab:
61
-
1. Select the **Subscription** in which to search for AI services (Azure OpenAI services or Azure AI Foundry projects). To get information about the deployments in a service, select the **deployments** link next to the service name.
65
+
1. Select the **Subscription** in which to search for AI services such as Azure AI Foundry or Azure OpenAI Service. To get information about the model deployments in a service, select the **deployments** link next to the service name.
62
66
:::image type="content" source="media/azure-ai-foundry-api/deployments.png" alt-text="Screenshot of deployments for an AI service in the portal.":::
63
67
1. Select an AI service.
64
68
1. Select **Next**.
65
69
1. On the **Configure API** tab:
66
70
1. Enter a **Display name** and optional **Description** for the API.
67
-
1. In **Path**, enter a path that your API Management instance uses to access the API endpoints.
71
+
1. In **Path**, enter a path that your API Management instance uses to access the deployment endpoint.
68
72
1. Optionally select one or more **Products** to associate with the API.
69
-
1. In **Client compatibility**, select either of the following based on the types of client you intend to support:
70
-
***Azure OpenAI** - Clients call the model deployment using the OpenAI API format. Select this option if you use only Azure OpenAI deployments.
71
-
***Azure AI** - Clients call the model deployment by passing
72
-
1. In **Access key**, optionally enter the authorization header name and API key used to access the LLM API.
73
+
1. In **Client compatibility**, select either of the following based on the types of client you intend to support. See [AI service options](#ai-service-options) for more information.
74
+
***Azure OpenAI** - Select this option if your deployment only includes Azure OpenAI Service model deployments.
75
+
***Azure AI** - Select this option if your deployment includes other models available through Azure AI Foundry.
73
76
1. Select **Next**.
77
+
78
+
:::image type="content" source="media/azure-ai-foundry-api/client-compatibility.png" alt-text="Screenshot of AI Foundry API configuration in the portal.":::
79
+
74
80
1. On the **Manage token consumption** tab, optionally enter settings or accept defaults that define the following policies to help monitor and manage the API:
@@ -81,19 +87,19 @@ On the **AI content safety**, optionally enter settings or accept defaults to co
81
87
1. Select **Review**.
82
88
1. After settings are validated, select **Create**.
83
89
84
-
## Test the LLM API
90
+
## Test the AI API
85
91
86
-
To ensure that your LLM API is working as expected, test it in the API Management test console.
92
+
To ensure that your AI API is working as expected, test it in the API Management test console.
87
93
1. Select the API you created in the previous step.
88
94
1. Select the **Test** tab.
89
-
1. Select an operation that's compatible with the model in the LLM API.
95
+
1. Select an operation that's compatible with the model in the lanaguage API.
90
96
The page displays fields for parameters and headers.
91
-
1. Enter parameters and headers as needed. Depending on the operation, you may need to configure or update a **Request body**.
97
+
1. Enter parameters and headers as needed. Depending on the operation, you might need to configure or update a **Request body**.
92
98
> [!NOTE]
93
99
> In the test console, API Management automatically populates an **Ocp-Apim-Subscription-Key** header, and configures the subscription key of the built-in [all-access subscription](api-management-subscriptions.md#all-access-subscription). This key enables access to every API in the API Management instance. Optionally display the **Ocp-Apim-Subscription-Key** header by selecting the "eye" icon next to the **HTTP Request**.
94
100
1. Select **Send**.
95
101
96
-
When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your Azure OpenAI API token consumption.
102
+
When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your language model token consumption.
This article shows two options to import an [Azure OpenAI Service](/azure/ai-services/openai/overview) API into an Azure API Management instance as a REST API:
17
+
You can import AI model endpoints deployed in [Azure OpenAI Service](/azure/ai-services/openai/overview) to your API Management instance as a REST API. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
18
+
19
+
20
+
This article shows two options to import an Azure OpenAI API into an Azure API Management instance as a REST API:
18
21
19
22
-[Import an Azure OpenAI API directly from Azure OpenAI Service](#option-1-import-api-from-azure-openai-service) (recommended)
20
23
-[Download and add the OpenAPI specification](#option-2-add-an-openapi-specification-to-api-management) for Azure OpenAI and add it to API Management as an OpenAPI API.
21
24
22
25
Learn more about managing AI APIs in API Management:
23
26
24
-
*[Generative AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
27
+
*[AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
25
28
26
29
## Prerequisites
27
30
@@ -134,7 +137,7 @@ To ensure that your Azure OpenAI API is working as expected, test it in the API
134
137
* `deployment-id` - the ID of a deployment in the Azure OpenAI service
135
138
* `api-version` - a valid Azure OpenAI API version, such as the API version you selected when you imported the API.
136
139
:::image type="content" source="media/azure-openai-api-from-specification/test-azure-openai-api.png" alt-text="Screenshot of testing an Azure OpenAI Service API in the portal." lightbox="media/azure-openai-api-from-specification/test-azure-openai-api.png" :::
137
-
1. Enter other parameters and headers as needed. Depending on the operation, you may need to configure or update a **Request body**.
140
+
1. Enter other parameters and headers as needed. Depending on the operation, you might need to configure or update a **Request body**.
138
141
> [!NOTE]
139
142
> In the test console, API Management automatically populates an **Ocp-Apim-Subscription-Key** header, and configures the subscription key of the built-in [all-access subscription](api-management-subscriptions.md#all-access-subscription). This key enables access to every API in the API Management instance. Optionally display the **Ocp-Apim-Subscription-Key** header by selecting the "eye" icon next to the **HTTP Request**.
You can import AI model endpoints deployed outside Azure to your API Management instance as a REST API. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
18
18
19
19
Learn more about managing AI APIs in API Management:
20
20
21
21
*[Generative AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
22
22
23
+
## Language model API types
24
+
25
+
API Management supports two types of self-hosted language model APIs. The option you select determines how clients call the API and how the API Management instance routes requests to the AI service.
26
+
27
+
***OpenAI-compatible** - Self-hosted model endpoints that are compatible with OpenAI's API. Examples include models exposed by inference providers such as [Hugging Face Text Generation Inference (TGI)](https://huggingface.co/docs/text-generation-inference/en/index).
28
+
29
+
API Management configures an OpenAI-compatible chat completions endpoint. Clients call the deployment at an `/openai` endpoint such as `/openai/deployments/my-deployment/chat/completions`.
30
+
31
+
***Passthrough** - Other self-hosted model endpoints that aren't compatible with OpenAI's API. Examples include models deployed in [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) or other providers.
32
+
33
+
API Management configures wildcard operations for common HTTP verbs. Clients can append paths to the wildcard operations, and API Management passes requests to the backend.
34
+
23
35
## Prerequisites
24
36
25
37
- An existing API Management instance. [Create one if you haven't already](get-started-create-service-instance.md).
26
-
- A self-hosted LLM with an API endpoint. You can use an OpenAI-compatible LLM that's exposed by an inference provider such as [Hugging Face Text Generation Inference (TGI)](https://huggingface.co/docs/text-generation-inference/en/index). Alternatively, you can access an LLM through a provider such as [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html).
27
-
> [!NOTE]
28
-
> API Management policies such as [llm-token-limit](llm-token-limit-policy.md) and [llm-emit-token-metric](llm-emit-token-metric-policy.md) are supported for APIs available through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api) or with OpenAI-compatible models served through third-party inference providers.
38
+
- A self-hosted (non-Azure-hosted) language model deployment with an API endpoint.
29
39
30
40
31
41
## Import language model API using the portal
32
42
33
-
Use the following steps to import an LLM API directly to API Management.
43
+
Use the following steps to import a language model API to API Management.
Depending on the API type you select to import, API Management automatically configures different operations to call the API:
38
-
39
-
***OpenAI-compatible API** - An operation for the LLM API's chat completion endpoint
40
-
***Passthrough API** - Wildcard operations for standard verbs `GET`, `HEAD`, `OPTIONS`, and `TRACK`. When you call the API, append any required path or parameters to the API request to pass a request to an LLM API endpoint.
41
-
42
-
For an OpenAI-compatible API, you can optionally configure policies to help you monitor and manage the API.
43
-
44
-
To import an LLM API to API Management:
47
+
To import a language model API to API Management:
45
48
46
49
1. In the [Azure portal](https://portal.azure.com), navigate to your API Management instance.
47
50
1. In the left menu, under **APIs**, select **APIs** > **+ Add API**.
@@ -54,9 +57,12 @@ To import an LLM API to API Management:
54
57
1. Enter the **URL** to the LLM API endpoint.
55
58
1. Optionally select one or more **Products** to associate with the API.
56
59
1. In **Path**, append a path that your API Management instance uses to access the LLM API endpoints.
57
-
1. In **Type**, select either **Create OpenAI API** or **Create a passthrough API**.
60
+
1. In **Type**, select either **Create OpenAI API** or **Create a passthrough API**. See [Language model API types](#language-model-api-types) for more information.
58
61
1. In **Access key**, optionally enter the authorization header name and API key used to access the LLM API.
59
62
1. Select **Next**.
63
+
64
+
:::image type="content" source="media/openai-compatible-llm-api/configure-api.png" alt-text="Screenshot of language model API configuration in the portal.":::
65
+
60
66
1. On the **Manage token consumption** tab, optionally enter settings or accept defaults that define the following policies to help monitor and manage the API:
@@ -74,12 +80,12 @@ To ensure that your LLM API is working as expected, test it in the API Managemen
74
80
1. Select the **Test** tab.
75
81
1. Select an operation that's compatible with the model in the LLM API.
76
82
The page displays fields for parameters and headers.
77
-
1. Enter parameters and headers as needed. Depending on the operation, you may need to configure or update a **Request body**.
83
+
1. Enter parameters and headers as needed. Depending on the operation, you might need to configure or update a **Request body**.
78
84
> [!NOTE]
79
85
> In the test console, API Management automatically populates an **Ocp-Apim-Subscription-Key** header, and configures the subscription key of the built-in [all-access subscription](api-management-subscriptions.md#all-access-subscription). This key enables access to every API in the API Management instance. Optionally display the **Ocp-Apim-Subscription-Key** header by selecting the "eye" icon next to the **HTTP Request**.
80
86
1. Select **Send**.
81
87
82
-
When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your Azure OpenAI API token consumption.
88
+
When the test is successful, the backend responds with a successful HTTP response code and some data. Appended to the response is token usage data to help you monitor and manage your language model token consumption.
0 commit comments