Skip to content

Commit 1a0a258

Browse files
daisyfaithaumakathaylMaddy-Cloudflare
authored
[AI Gateway] Added custom costs (#16689)
* Added custom costs * Update custom-costs.mdx 1) lowercased "costs" in title to match titles of other pages in Configuration section 2) Removed section on unsupported costs. Reason being, we will never know the full extent of costs we do not support, given all the changes, new models, etc. in the AI space. Instead, we will use the "Supported Providers" section in the Costs page (https://developers.cloudflare.com/ai-gateway/observability/costs/) to explain what we CAN support * Update src/content/docs/ai-gateway/configuration/custom-costs.mdx Grammar Co-authored-by: Maddy <[email protected]> * Update src/content/docs/ai-gateway/configuration/custom-costs.mdx Co-authored-by: Maddy <[email protected]> * Minor errors fixed * Changed custom costs note --------- Co-authored-by: Kathy <[email protected]> Co-authored-by: Maddy <[email protected]>
1 parent 08b627e commit 1a0a258

File tree

1 file changed

+47
-0
lines changed

1 file changed

+47
-0
lines changed
Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
pcx_content_type: configuration
3+
title: Custom costs
4+
sidebar:
5+
order: 3
6+
head: []
7+
description: Override default or public model costs on a per-request basis.
8+
---
9+
10+
import { TabItem, Tabs } from "~/components";
11+
12+
AI Gateway allows you to set custom costs at the request level. By using this feature, the cost metrics can accurately reflect your unique pricing, overriding the default or public model costs.
13+
14+
:::note[Note]
15+
16+
Custom costs will only apply to requests that pass tokens in their response. Requests without token information will not have costs calculated.
17+
18+
:::
19+
20+
## Custom cost
21+
22+
To add custom costs to your API requests, use the `cf-aig-custom-cost` header. This header enables you to specify the cost per token for both input (tokens sent) and output (tokens received).
23+
24+
- **per_token_in**: The negotiated input token cost (per token).
25+
- **per_token_out**: The negotiated output token cost (per token).
26+
27+
There is no limit to the number of decimal places you can include, ensuring precise cost calculations, regardless of how small the values are.
28+
29+
Custom costs will appear in the logs with an underline, making it easy to identify when custom pricing has been applied.
30+
31+
In this example, if you have a negotiated price of $1 per million input tokens and $2 per million output tokens, include the `cf-aig-custom-cost` header as shown below.
32+
33+
```bash title="Request with custom cost"
34+
curl https://gateway.ai.cloudflare.com/v1/{{account_id}}/{{gateway_id}}/openai/chat/completions \
35+
--header 'Authorization: Bearer $TOKEN' \
36+
--header 'Content-Type: application/json' \
37+
--header 'cf-aig-custom-cost: {{"per_token_in":0.000001,"per_token_out":0.000002}}' \
38+
--data ' {{
39+
"model": "gpt-4o-mini",
40+
"messages": [
41+
{{
42+
"role": "user",
43+
"content": "When is Cloudflare’s Birthday Week?"
44+
}}
45+
]
46+
}}'
47+
```

0 commit comments

Comments
 (0)