Merge pull request #295255 from dlepow/ailog

[APIM] LLM logging

Author: v-dirichards

articles/api-management/TOC.yml

@@ -624,10 +624,6 @@
   - name: Azure Policy built-ins
     displayName: samples, policies, definitions
     href: ./policy-reference.md
-  - name: Gateway log schema
-    href: gateway-log-schema-reference.md
-  - name: Developer portal audit log schema
-    href: developer-portal-audit-log-schema-reference.md
   - name: Event Grid events schema
     href: ../event-grid/event-schema-api-management.md?toc=/azure/api-management/toc.json&bc=/azure/api-management/breadcrumb/toc.json
   - name: Virtual network configuration
@@ -636,8 +632,6 @@
     href: self-hosted-gateway-settings-reference.md
   - name: Self-hosted gateway Azure Arc configuration
     href: self-hosted-gateway-arc-reference.md
-  - name: Diagnostic logs settings
-    href: diagnostic-logs-reference.md
 - name: Resources
   items:
   - name: FAQ

articles/api-management/developer-portal-audit-log-schema-reference.md

@@ -104,6 +104,16 @@ In API Management, enable semantic caching by using Azure Redis Enterprise, Azur
 > [!TIP]
 > To enable semantic caching for other LLM APIs, API Management provides the equivalent [llm-semantic-cache-store-policy](llm-semantic-cache-store-policy.md) and [llm-semantic-cache-lookup-policy](llm-semantic-cache-lookup-policy.md) policies.
 
+## Logging token usage, prompts, and completions
+
+Enable a [diagnostic setting](monitor-api-management.md#enable-diagnostic-setting-for-azure-monitor-logs) in your API Management instance to log requests processed by the gateway for large language model REST APIs. For each request, data is sent to Azure Monitor including token usage (prompt tokens, completion tokens, and total tokens), name of the model used, and optionally the request and response messages (prompt and completion). Large requests and responses are split into multiple log entries that are sequentially numbered for later reconstruction if needed.
+
+The API Management administrator can use LLM gateway logs along with API Management gateway logs for scenarios such as the following:
+
+* **Calculate usage for billing** - Calculate usage metrics for billing based on the number of tokens consumed by each application or API consumer (for example, segmented by subscription ID or IP address).
+* **Inspect messages** - To help with debugging or auditing, inspect and analyze prompts and completions.
+
+Learn more about [monitoring API Management with Azure Monitor](monitor-api-management.md).
 
 ## Content safety policy