You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can import self-hosted AI model endpoints to your API Management instance as APIs. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
17
+
You can import OpenAI-compatible language model endpoints to your API Management instance as APIs. For example, you might want to manage an LLM that you self-host, or that's hosted on an inference provider other than Azure AI services. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
18
18
19
19
Learn more about managing AI APIs in API Management:
20
20
21
21
*[Generative AI gateway capabilities in Azure API Management](genai-gateway-capabilities.md)
22
22
23
23
## Language model API types
24
24
25
-
API Management supports two types of self-hosted language model APIs. Choose the option suitable for your model deployment. The option determines how clients call the API and how the API Management instance routes requests to the AI service.
25
+
API Management supports two types of language model APIs for this scenario. Choose the option suitable for your model deployment. The option determines how clients call the API and how the API Management instance routes requests to the AI service.
26
26
27
-
***OpenAI-compatible** - Self-hosted model endpoints that are compatible with OpenAI's API. Examples include certain models exposed by inference providers such as [Hugging Face Text Generation Inference (TGI)](https://huggingface.co/docs/text-generation-inference/en/index).
27
+
***OpenAI-compatible** - Language model endpoints that are compatible with OpenAI's API. Examples include certain models exposed by inference providers such as [Hugging Face Text Generation Inference (TGI)](https://huggingface.co/docs/text-generation-inference/en/index).
28
28
29
29
API Management configures an OpenAI-compatible chat completions endpoint.
30
30
31
-
***Passthrough** - Other self-hosted model endpoints that aren't compatible with OpenAI's API. Examples include models deployed in [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) or other providers.
31
+
***Passthrough** - Other language model endpoints that aren't compatible with OpenAI's API. Examples include models deployed in [Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) or other providers.
32
32
33
33
API Management configures wildcard operations for common HTTP verbs. Clients can append paths to the wildcard operations, and API Management passes requests to the backend.
34
34
35
35
## Prerequisites
36
36
37
37
- An existing API Management instance. [Create one if you haven't already](get-started-create-service-instance.md).
38
-
- A self-hosted (non-Azure-hosted) language model deployment with an API endpoint.
38
+
- A self-hosted or non-Azure-provided language model deployment with an API endpoint.
39
39
40
40
41
41
## Import language model API using the portal
42
42
43
43
44
-
To import a self-hosted language model API to API Management:
44
+
To import a language model API to API Management:
45
45
46
46
1. In the [Azure portal](https://portal.azure.com), navigate to your API Management instance.
47
47
1. In the left menu, under **APIs**, select **APIs** > **+ Add API**.
@@ -55,7 +55,7 @@ To import a self-hosted language model API to API Management:
55
55
1. Optionally select one or more **Products** to associate with the API.
56
56
1. In **Path**, append a path that your API Management instance uses to access the LLM API endpoints.
57
57
1. In **Type**, select either **Create OpenAI API** or **Create a passthrough API**. See [Language model API types](#language-model-api-types) for more information.
58
-
1. In **Access key**, optionally enter the authorization header name and API key used to access the LLM API.
58
+
1. In **Access key**, enter the authorization header name and API key used to access the LLM API, if required.
59
59
1. Select **Next**.
60
60
61
61
:::image type="content" source="media/openai-compatible-llm-api/configure-api.png" alt-text="Screenshot of language model API configuration in the portal.":::
0 commit comments