← All answers
Question
What are the best practices for AI agent observability?
Short answer
Trace every run end to end, record tool calls and cost on each step, review failures by segment instead of averages, add evaluations for quality, and alert on changes. Start with traces and add the rest as you need it.
In more detail
1. Trace first
A trace per run, with a span for each model call and tool call, answers most debugging questions.
2. Track cost and latency
Record tokens, cost and time on every step so a slow or expensive run is easy to spot.
3. Add evaluations
Score outputs on a sample of production runs and on a fixed dataset before each release.
4. Alert and gate
Alert on errors and cost spikes, and block a release that scores worse than the last one.
