refine OpenRouter cost tracking

Author: season179

.changeset/openrouter-cost-tracking.md

@@ -3,9 +3,9 @@
 '@tanstack/ai': minor
 ---
 
-- Add OpenRouter cost tracking. The `OpenRouterTextAdapter` now attaches the authoritative per-request USD cost to the `RUN_FINISHED` event under `usage.cost`, along with `usage.costDetails.upstreamInferenceCost` and `usage.costDetails.cacheDiscount` when present. Cost is sourced from OpenRouter's chat completion response itself (the field arrives in the trailing SSE chunk), so there is no extra HTTP request and no added end-of-stream latency.
-- Cost is always sourced from OpenRouter — never computed locally from token counts and a price table — because OpenRouter routes the same model id to different upstream providers (primary, fallback, BYOK) with different pricing, and applies cache discounts the SDK cannot reconstruct.
-- Cost is captured via the SDK's public `HTTPClient` response hook. The hook calls `Response.clone()` and parses the cloned body out-of-band to recover `usage.cost` and `usage.cost_details`, which the @openrouter/sdk Zod parser would otherwise strip (the schema doesn't declare those fields). The SDK's stream consumer is unaffected — both clones are read independently.
+- Add OpenRouter cost tracking. The `OpenRouterTextAdapter` now attaches OpenRouter's authoritative per-request cost amount to the `RUN_FINISHED` event under `usage.cost`, along with numeric/null fields from `usage.cost_details` under `usage.costDetails`. Cost is sourced from OpenRouter's chat completion response itself (the field arrives in the trailing SSE chunk), so there is no extra HTTP request and no added end-of-stream latency.
+- Cost is always sourced from OpenRouter — never computed locally from token counts and a price table — because OpenRouter routes the same model id to different upstream providers (primary, fallback, BYOK) with different pricing, and may expose provider-specific cost breakdowns the SDK cannot reconstruct.
+- Cost is captured via the SDK's public `HTTPClient` response hook. The hook calls `Response.clone()` and parses the cloned body out-of-band to recover `usage.cost` and `usage.cost_details`, which the @openrouter/sdk chat-completion parser would otherwise strip. The SDK's stream consumer is unaffected — both clones are read independently.
 - Custom `httpClient` values passed into the adapter are preserved: the adapter clones the caller's client (inheriting their fetcher, retries, tracing, and any pre-registered hooks) and appends the cost-capture hook to the clone, so the caller's original instance is never mutated and cost tracking still works when callers supply their own transport.
 - Defer the OpenRouter `RUN_FINISHED` emission until after the upstream stream fully drains, so token usage that arrives in a trailing usage-only chunk (the common case for OpenAI-compatible providers, where the final chunk has empty `choices`) is included in `usage` instead of being dropped.
 - Extend `RunFinishedEvent.usage` in `@tanstack/ai` with optional `cost` and `costDetails` fields. The middleware `UsageInfo` (consumed by `onUsage`) and `FinishInfo.usage` (consumed by `onFinish`) carry the same fields, so middleware authors can read cost without casts. The change is additive and backwards-compatible for adapters that don't populate cost.

README.md

@@ -46,7 +46,7 @@ A powerful, type-safe AI SDK for building AI-powered applications.
 - Isomorphic type-safe tools with server/client execution
 - **Enhanced integration with TanStack Start** - Share implementations between AI tools and server functions
 - **Observability events** - Structured, typed events for text, tools, image, speech, transcription, and video ([docs](./docs/guides/observability.md))
-- **Cost tracking** - Per-request USD cost on `RUN_FINISHED` for providers that report it (currently [OpenRouter](./docs/adapters/openrouter.md#cost-tracking))
+- **Cost tracking** - Per-request cost on `RUN_FINISHED` for providers that report it (currently [OpenRouter](./docs/adapters/openrouter.md#cost-tracking))
 
 ### <a href="https://tanstack.com/ai">Read the docs →</a>
 

docs/adapters/openrouter.md

@@ -117,9 +117,9 @@ OPENROUTER_API_KEY=sk-or-...
 
 ## Cost Tracking
 
-The OpenRouter adapter attaches the authoritative per-request cost to the `RUN_FINISHED` event under `usage.cost` (USD). OpenRouter [reports cost inline in every chat response](https://openrouter.ai/docs/use-cases/usage-accounting), so cost arrives in the same SSE stream as the model output — there is **no extra HTTP request** and **no added latency**.
+The OpenRouter adapter attaches the authoritative per-request cost to the `RUN_FINISHED` event under `usage.cost`. OpenRouter [reports cost inline in every chat response](https://openrouter.ai/docs/use-cases/usage-accounting), in credits, so cost arrives in the same SSE stream as the model output — there is **no extra HTTP request** and **no added latency**.
 
-Why we don't compute cost locally from tokens × price: OpenRouter routes the same model id to different upstream providers (primary, fallback, BYOK), each with different pricing, plus applies cache discounts and BYOK upstream costs. A static price table would silently drift and produce wrong numbers.
+Why we don't compute cost locally from tokens × price: OpenRouter routes the same model id to different upstream providers (primary, fallback, BYOK), each with different pricing, and may include cached-token pricing or BYOK upstream costs. A static price table would silently drift and produce wrong numbers.
 
 ```typescript
 import { chat } from "@tanstack/ai";
@@ -132,9 +132,8 @@ const stream = chat({
 
 for await (const chunk of stream) {
   if (chunk.type === "RUN_FINISHED") {
-    console.log("USD cost:", chunk.usage?.cost);
+    console.log("OpenRouter cost:", chunk.usage?.cost);
     console.log("Upstream inference cost:", chunk.usage?.costDetails?.upstreamInferenceCost);
-    console.log("Cache discount:", chunk.usage?.costDetails?.cacheDiscount);
   }
 }
 ```
@@ -203,4 +202,3 @@ const stream = chat({
 ```
 
 **Supported models:** all OpenRouter chat models. See [Provider Tools](../tools/provider-tools.md#which-models-support-which-tools).
-

packages/typescript/ai-openrouter/src/adapters/cost-capture.ts

@@ -1,11 +1,8 @@
 import { HTTPClient } from '@openrouter/sdk'
 
 export interface CostInfo {
-  cost?: number
-  costDetails?: {
-    upstreamInferenceCost?: number | null
-    cacheDiscount?: number | null
-  }
+  cost: number
+  costDetails?: Record<string, number | null | undefined>
 }
 
 interface CostEntry {
@@ -310,42 +307,43 @@ function extractDataPayload(event: string): string | undefined {
 function safeParseJson(text: string): Record<string, unknown> | undefined {
   try {
     const v = JSON.parse(text)
-    return v && typeof v === 'object' ? (v as Record<string, unknown>) : undefined
+    return v && typeof v === 'object'
+      ? (v as Record<string, unknown>)
+      : undefined
   } catch {
     return undefined
   }
 }
 
-function pickNumberOrNull(
-  obj: Record<string, unknown> | undefined,
-  key: string,
-): number | null | undefined {
-  if (!obj) return undefined
-  const v = obj[key]
-  if (typeof v === 'number') return v
-  if (v === null) return null
-  return undefined
+function toCamelCase(key: string): string {
+  return key.replace(/_([a-z])/g, (_, char: string) => char.toUpperCase())
+}
+
+function extractCostDetails(
+  details: Record<string, unknown> | undefined,
+): CostInfo['costDetails'] | undefined {
+  if (!details) return undefined
+  const result: NonNullable<CostInfo['costDetails']> = {}
+  for (const [key, value] of Object.entries(details)) {
+    if (typeof value === 'number' || value === null) {
+      result[toCamelCase(key)] = value
+    }
+  }
+  return Object.keys(result).length > 0 ? result : undefined
 }
 
 function extractCostFromUsage(
   usage: Record<string, unknown>,
 ): CostInfo | undefined {
-  // `cost` is the authoritative per-request USD total. Emitting `costDetails`
-  // without it would surface a breakdown that can't be reconciled against a
-  // total — callers could misread it as the bill.
+  // `cost` is the authoritative per-request total reported by OpenRouter.
+  // Emitting `costDetails` without it would surface a breakdown that can't be
+  // reconciled against a total — callers could misread it as the bill.
   const cost = typeof usage.cost === 'number' ? usage.cost : undefined
   if (cost === undefined) return undefined
   const details = usage.cost_details as Record<string, unknown> | undefined
-  const upstream = pickNumberOrNull(details, 'upstream_inference_cost')
-  const cacheDiscount = pickNumberOrNull(details, 'cache_discount')
-  const hasDetails = upstream !== undefined || cacheDiscount !== undefined
+  const costDetails = extractCostDetails(details)
   return {
     cost,
-    ...(hasDetails && {
-      costDetails: {
-        ...(upstream !== undefined && { upstreamInferenceCost: upstream }),
-        ...(cacheDiscount !== undefined && { cacheDiscount }),
-      },
-    }),
+    ...(costDetails && { costDetails }),
   }
 }

packages/typescript/ai-openrouter/src/adapters/text.ts

@@ -164,66 +164,66 @@ export class OpenRouterTextAdapter<
         if (chunk.id) responseId = chunk.id
         if (chunk.model) currentModel = chunk.model
 
-          // Emit RUN_STARTED on first chunk
-          if (!aguiState.hasEmittedRunStarted) {
-            aguiState.hasEmittedRunStarted = true
-            yield asChunk({
-              type: 'RUN_STARTED',
-              runId: aguiState.runId,
-              threadId: aguiState.threadId,
-              model: currentModel || options.model,
-              timestamp,
-            })
-          }
+        // Emit RUN_STARTED on first chunk
+        if (!aguiState.hasEmittedRunStarted) {
+          aguiState.hasEmittedRunStarted = true
+          yield asChunk({
+            type: 'RUN_STARTED',
+            runId: aguiState.runId,
+            threadId: aguiState.threadId,
+            model: currentModel || options.model,
+            timestamp,
+          })
+        }
 
-          if (chunk.error) {
-            // Emit AG-UI RUN_ERROR
-            yield asChunk({
-              type: 'RUN_ERROR',
-              runId: aguiState.runId,
-              model: currentModel || options.model,
-              timestamp,
+        if (chunk.error) {
+          // Emit AG-UI RUN_ERROR
+          yield asChunk({
+            type: 'RUN_ERROR',
+            runId: aguiState.runId,
+            model: currentModel || options.model,
+            timestamp,
+            message: chunk.error.message || 'Unknown error',
+            code: String(chunk.error.code),
+            error: {
               message: chunk.error.message || 'Unknown error',
               code: String(chunk.error.code),
-              error: {
-                message: chunk.error.message || 'Unknown error',
-                code: String(chunk.error.code),
-              },
-            })
-            continue
-          }
+            },
+          })
+          continue
+        }
 
-          for (const choice of chunk.choices) {
-            yield* this.processChoice(
-              choice,
-              toolCallBuffers,
-              {
-                id: responseId || this.generateId(),
-                model: currentModel,
-                timestamp,
-              },
-              { reasoning: accumulatedReasoning, content: accumulatedContent },
-              (r, c) => {
-                accumulatedReasoning = r
-                accumulatedContent = c
-              },
-              chunk.usage,
-              aguiState,
-            )
-          }
+        for (const choice of chunk.choices) {
+          yield* this.processChoice(
+            choice,
+            toolCallBuffers,
+            {
+              id: responseId || this.generateId(),
+              model: currentModel,
+              timestamp,
+            },
+            { reasoning: accumulatedReasoning, content: accumulatedContent },
+            (r, c) => {
+              accumulatedReasoning = r
+              accumulatedContent = c
+            },
+            chunk.usage,
+            aguiState,
+          )
+        }
 
-          // Capture usage from a trailing `choices: []` chunk that the
-          // choice loop above would have skipped. OpenRouter (and other
-          // OpenAI-compatible streams) often report final token counts in
-          // a terminal chunk with no choices, after `finishReason` was
-          // delivered on an earlier chunk.
-          if (chunk.usage && !aguiState.deferredUsage) {
-            aguiState.deferredUsage = {
-              promptTokens: chunk.usage.promptTokens || 0,
-              completionTokens: chunk.usage.completionTokens || 0,
-              totalTokens: chunk.usage.totalTokens || 0,
-            }
+        // Capture usage from a trailing `choices: []` chunk that the
+        // choice loop above would have skipped. OpenRouter (and other
+        // OpenAI-compatible streams) often report final token counts in
+        // a terminal chunk with no choices, after `finishReason` was
+        // delivered on an earlier chunk.
+        if (chunk.usage && !aguiState.deferredUsage) {
+          aguiState.deferredUsage = {
+            promptTokens: chunk.usage.promptTokens || 0,
+            completionTokens: chunk.usage.completionTokens || 0,
+            totalTokens: chunk.usage.totalTokens || 0,
           }
+        }
       }
 
       // Emit RUN_FINISHED after the stream ends so we capture usage from
@@ -376,11 +376,11 @@ export class OpenRouterTextAdapter<
     // trailing usage chunk), emit no usage at all — even if `costInfo` was
     // captured. Synthesizing zero-token counts alongside a non-zero cost
     // would feed billing/telemetry a "successful run with zero tokens but
-    // $X cost" signal, which is worse than an absent usage payload.
+    // nonzero cost" signal, which is worse than an absent usage payload.
     if (!usage) return undefined
     return {
       ...usage,
-      ...(costInfo?.cost !== undefined && { cost: costInfo.cost }),
+      ...(costInfo && { cost: costInfo.cost }),
       ...(costInfo?.costDetails && { costDetails: costInfo.costDetails }),
     }
   }