Track model cost and latency

Cost analysis is most useful beside quality and latency. Send the model identifier and token usage on generation spans, then add environment and release metadata so the dashboard can compare deployments.

Use the trace waterfall to find slow retrieval or tool calls, and the cost view to find expensive models or prompts. Optimize only after checking that the change preserves the quality signals tracked by evaluation.

Record tokens and cost

On a manual span, record what the call used with step.set:

await span(
  "chat",
  async (step) => {
    const reply = await callModel(question);
    step.set({
      model: "your-model-id",
      inputTokens: reply.usage.input,
      outputTokens: reply.usage.output,
      costUsd: priceFor(reply.usage),
    });
    return reply.text;
  },
  { type: "llm" },
);

with tracify.span("chat", type="llm", input=question) as step:
    reply = call_model(question)
    step.set(
        model="your-model-id",
        input_tokens=reply.usage.input,
        output_tokens=reply.usage.output,
        cost_usd=price_for(reply.usage),
    )

priceFor and price_for are your own functions using your current provider rates. Tracify never guesses a price: if you do not pass cost, it stays zero. With wrapped provider clients, supply calculateCost once instead. See OpenAI, Anthropic and LangGraph.