Skip to content

Commit ee2a20c

Browse files
author
gitName
committed
add to genai capabilities
1 parent 9ff4efc commit ee2a20c

File tree

2 files changed

+11
-1
lines changed

2 files changed

+11
-1
lines changed

articles/api-management/genai-gateway-capabilities.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -104,6 +104,16 @@ In API Management, enable semantic caching by using Azure Redis Enterprise or an
104104
> [!TIP]
105105
> To enable semantic caching for LLM APIs available through the Azure AI Model Inference API, API Management provides the equivalent [llm-semantic-cache-store-policy](llm-semantic-cache-store-policy.md) and [llm-semantic-cache-lookup-policy](llm-semantic-cache-lookup-policy.md) policies.
106106
107+
## Azure Monitor logging
108+
109+
Enable a [diagnostic setting](monitor-api-management.md#enable-diagnostic-setting-for-azure-monitor-logs) in your API Management instance to send logs to Azure Monitor for requests processed by the gateway for large language model REST APIs. For each request, data is logged including token usage (prompt tokens, completion tokens, and total tokens), name of the model used, and optionally the request message and response message. Large requests and responses are split into multiple log entries that are sequentially numbered for later reconstruction if needed.
110+
111+
The API Management administrator can use LLM gateway logs along with API Management gateway logs for scenarios such as the following:
112+
113+
* **Calculate usage for billing** - Calculate usage metrics for billing based on the number of tokens consumed by each application or API consumer (for example, segmented by subscription ID or IP address).
114+
* **Inspect messages** - Inspect and analyze prompts and completions for debugging or auditing.
115+
116+
Learn more about [monitoring API Management with Azure Monitor](monitor-api-management.md).
107117

108118
## Labs and samples
109119

articles/api-management/visual-studio-code-tutorial.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ ms.service: azure-api-management
55
author: dlepow
66
ms.author: danlep
77
ms.topic: tutorial
8-
ms.date: 02/24/2025
8+
ms.date: 02/20/2025
99
ms.custom: devdivchpfy22
1010
---
1111

0 commit comments

Comments
 (0)