Product / Cost dashboard
Know where the money went.
Spend broken down by day, run, model and tool, with retries and fallbacks counted separately. Find the call that doubled your bill, then open the exact run behind it.
- Granularity
- Per step
- Retry cost
- Separated
- Models
- Ranked
Explain the bill
Follow a spike to its cause.
Start with the release or time range where spend changed rather than an all-time average.
Compare models, tools, retries, and fallback cost to identify the few operations worth inspecting.
Confirm whether a cost pattern is necessary work, duplicated work, or a policy problem before changing it.
Turn a rising invoice into a short list of things to fix.
Cost over time
Read daily spend and tokens across a selected range without losing the run-level records behind each aggregate.
Model attribution
Compare calls, latency, and total cost by model to find expensive defaults or unexpected routing behavior.
Tool attribution
See tool activity and cost alongside model usage, including workflows where external calls dominate runtime.
Retry and fallback overhead
Separate recoverability cost from primary execution so resilience policy can be tuned with evidence.
Who it is for
One retry loop can triple a bill.
Teams comparing model choices, investigating cost regressions, or setting practical budgets for production agents.
Instrument it
Put a price on every call.
Record cost where the call happens. Spend then rolls up by day, run, model and tool, with retries counted on their own.
$ npm install tracify-sdk
import { traceAgent, llmCall } from "tracify-sdk";
export const answer = traceAgent(async (question: string) => {
const reply = await callModel(question);
await llmCall({
input: question,
output: reply.text,
modelId: "gpt-4o",
costUsd: 0.002,
latencyMs: 800,
});
return reply.text;
}, { environment: "production", release: "2.14" });Questions people ask.
How do I track LLM API costs?
Track LLM API costs by attaching cost data to each model and tool call at the trace level, then aggregating by day, model, and run so spend can be traced back to its source.
Why are my LLM costs so high?
The most common causes are expensive default models, unnecessary retries or fallback chains, and tool calls that run more often than the workflow requires. A cost dashboard broken down by model and tool ranks these contributors.
How do I reduce OpenAI API costs?
Compare cost by model and route lower-stakes calls to cheaper models, cap retries with runtime policy, and confirm expensive changes against real trace evidence before rolling them out broadly.
What is the difference between primary cost and retry cost?
Primary cost is the spend from the intended execution path. Retry and fallback cost is the additional spend from recovery behavior, and separating the two shows how much resilience policy is actually costing.
Can I compare AI agent costs by model?
Yes. Cost Dashboard breaks down calls, latency, and total spend by model so you can identify which model choices or routing decisions are driving the bill.
See your own runs like this.