Use case / Chatbots
See where the conversation lost the thread.
A chatbot answers one message at a time, but the problem is often ten messages back. Trace every turn with the history it was given, and a drifting reply is a step you can open.
- Turns
- Traced
- History
- Visible
- Fallbacks
- Counted
Conversation record
The turn where it lost the thread.
Each conversation is one trace, with every turn inside it.
See exactly which messages were loaded for this reply.
Trimmed history is visible, so a forgotten detail has a cause.
The reply sits next to the context that produced it.
Compare closed conversations to find where people give up.
- 01ok
Message received
“Can I change the address on my order?”
- 02ok
History loaded
Eleven earlier messages, 3,100 tokens.
- 03drift
Context trimmed
The order number fell out of the window.
- 04drift
Reply drifts off topic
The bot asks for an order number again.
- 05ok
Conversation closed
The customer left after four turns.
What Tracify watches.
Turn by turn context
Recorded on every run, so it can be searched, compared and alerted on.
Answer latency
Recorded on every run, so it can be searched, compared and alerted on.
Fallback replies
Recorded on every run, so it can be searched, compared and alerted on.
Cost per conversation
Recorded on every run, so it can be searched, compared and alerted on.
The failure to catch
A friendly reply that forgot what the customer said three messages ago.
Trace every turn of a customer-facing chatbot, with the context it was given, the answer it wrote and the moment it drifted.
Instrument it
Trace every turn, not just the chat.
Open one trace per conversation and one span per turn, and the history each reply used is stored next to it.
$ npm install tracify-sdk
import { span, traceAgent } from "tracify-sdk";
export const reply = traceAgent(async (history: string[], message: string) => {
const context = await span("load-history", () => trimHistory(history), { type: "tool", input: message });
return span("reply", () => callModel(context, message), { type: "llm" });
}, { environment: "production" });