Skip to content

Commit 9055656

Browse files
author
gitName
committed
edits
1 parent a4ead18 commit 9055656

File tree

3 files changed

+8
-7
lines changed

3 files changed

+8
-7
lines changed

articles/api-management/genai-gateway-capabilities.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -104,14 +104,14 @@ In API Management, enable semantic caching by using Azure Redis Enterprise or an
104104
> [!TIP]
105105
> To enable semantic caching for LLM APIs available through the Azure AI Model Inference API, API Management provides the equivalent [llm-semantic-cache-store-policy](llm-semantic-cache-store-policy.md) and [llm-semantic-cache-lookup-policy](llm-semantic-cache-lookup-policy.md) policies.
106106
107-
## Azure Monitor logging
107+
## Logging token usage, prompts, and completions
108108

109-
Enable a [diagnostic setting](monitor-api-management.md#enable-diagnostic-setting-for-azure-monitor-logs) in your API Management instance to send logs to Azure Monitor for requests processed by the gateway for large language model REST APIs. For each request, data is logged including token usage (prompt tokens, completion tokens, and total tokens), name of the model used, and optionally the request message and response message. Large requests and responses are split into multiple log entries that are sequentially numbered for later reconstruction if needed.
109+
Enable a [diagnostic setting](monitor-api-management.md#enable-diagnostic-setting-for-azure-monitor-logs) in your API Management instance to log requests processed by the gateway for large language model REST APIs. For each request, data is sent to Azure Monitor including token usage (prompt tokens, completion tokens, and total tokens), name of the model used, and optionally the request and response messages (prompt and completion). Large requests and responses are split into multiple log entries that are sequentially numbered for later reconstruction if needed.
110110

111111
The API Management administrator can use LLM gateway logs along with API Management gateway logs for scenarios such as the following:
112112

113113
* **Calculate usage for billing** - Calculate usage metrics for billing based on the number of tokens consumed by each application or API consumer (for example, segmented by subscription ID or IP address).
114-
* **Inspect messages** - Inspect and analyze prompts and completions for debugging or auditing.
114+
* **Inspect messages** - To help with debugging or auditing, inspect and analyze prompts and completions.
115115

116116
Learn more about [monitoring API Management with Azure Monitor](monitor-api-management.md).
117117

articles/api-management/monitor-api-management.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ title: Monitor Azure API Management
33
description: Learn how to monitor Azure API Management using Azure Monitor, including data collection, analysis, and alerting.
44
ms.date: 01/06/2025
55
ms.custom: horz-monitor
6-
ms.topic: concept-article
6+
ms.topic: how-to
77
author: dlepow
88
ms.author: danlep
99
ms.service: azure-api-management
@@ -197,4 +197,5 @@ To see how to set up an alert rule in Azure API Management, see [Set up an alert
197197
## Related content
198198

199199
- [API Management monitoring data reference](monitor-api-management-reference.md)
200+
- [Tutorial: Monitor published APIs](api-management-howto-use-azure-monitor.md)
200201
- [Monitoring Azure resources with Azure Monitor](/azure/azure-monitor/essentials/monitor-azure-resource)

includes/api-management-log-entry-size-limit.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,6 @@ ms.author: danlep
77
---
88

99
> [!IMPORTANT]
10-
> API Management enforces a 32 KB limit for the size of log entries sent to Azure Monitor. The behavior when the logged data exceeds the limit depends on the log category and the data attributes that are logged:
11-
> * **API Management Gateway logs** - Logged request or response payloads in a log entry, if collected, can be up to 8,192 bytes each. If the combined size of the logged attributes exceeds 32 KB, API Management trims the entry by removing all body and trace content.
12-
> * **LLM Gateway logs** - Request or response messages, if collected, are logged up to 32 KB; messages larger than 32 KB are split and logged in 32 KB chunks including sequence numbers for later reconstruction. Request messages and response messages can't exceed 2 MB each.
10+
> API Management enforces a 32 KB limit for the size of log entries sent to Azure Monitor. The behavior when a log entry exceeds the limit depends on the log category and the data attributes that are logged:
11+
> * **API Management Gateway logs** - Logged request or response payloads in a log entry, if collected, can be up to 8,192 bytes each. If the combined size of the attributes in an entry exceeds 32 KB, API Management trims the entry by removing all body and trace content.
12+
> * **LLM Gateway logs** - LLM request or response messages up to 32 KB in size, if collected, are sent in a single entry. Messages larger than 32 KB are split and logged in 32 KB chunks with sequence numbers for later reconstruction. Request messages and response messages can't exceed 2 MB each.

0 commit comments

Comments
 (0)