Skip to content

Commit 42089d1

Browse files
committed
feat(costs): add token/cost estimator with env-configurable prices; track_usage enabled; README documents cost tokens; maintain 1.0 success with demos
1 parent 572460b commit 42089d1

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

README.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,16 @@ Code references (discoverability)
135135
- Model: `gpt-4o-mini`, N=30
136136
- Before (no demos): success_rate 1.00; avg_latency_sec ~0.188; avg_lm_calls 3.33; avg_tool_calls 1.17; avg_steps 3.17
137137
- After (compiled demos loaded): success_rate 1.00; avg_latency_sec ~0.188; avg_lm_calls 3.33; avg_tool_calls 1.17; avg_steps 3.17
138-
Notes: For this small dataset, demos neither help nor hurt. Seed demos from your real tasks for measurable gains.
138+
Notes: For this small dataset, demos neither help nor hurt. For larger flows, compile demos from your real tasks.
139+
140+
### Cost & Tokens
141+
- The agent aggregates token counts and cost (when available). If provider usage isn’t exposed, it estimates tokens from prompts/outputs and computes cost using env prices.
142+
- Set env prices for OpenAI models (USD per 1K tokens):
143+
```bash
144+
export OPENAI_INPUT_PRICE_PER_1K=0.005 # example
145+
export OPENAI_OUTPUT_PRICE_PER_1K=0.015 # example
146+
```
147+
The eval script will include `avg_cost_usd`. Defaults to 0 if prices aren’t set.
139148
140149
## Optimize (Teleprompting)
141150
- Compile optimized few-shot demos for the OpenAI `PlanWithTools` planner and save to JSON:

0 commit comments

Comments
 (0)