Feedback and scores

Cost and latency tell you how an agent ran. Feedback and scores tell you whether it was any good. Attach them to the run that produced the answer and they appear on the trace, where you can filter by them and use them in evaluations.

You need the run id of the trace and a client created with your projectId. The calls use the same API key as ingestion and never throw into your agent if delivery fails. Without a project id nothing is sent: the TypeScript client skips the call silently, and the Python client prints a warning.

Record end-user feedback

import { TracifyClient } from "tracify-sdk";

const client = new TracifyClient({
  apiKey: process.env.TRACIFY_API_KEY,
  projectId: "your-project-id",
});

client.feedback(runId, true, { kind: "thumb", comment: "Helpful answer" });
client.feedback(runId, 4, { kind: "star", reason: "Correct but slow" });

from tracify import TracifyClient

client = TracifyClient(project_id="your-project-id")

client.feedback(run_id, True, kind="thumb", comment="Helpful answer")
client.feedback(run_id, 4, kind="star", reason="Correct but slow")

kind is thumb, star or text. If you leave it out, it is inferred from the value: a boolean is a thumb, a number is a star, and a string is text. Pass dedupeKey (dedupe_key in Python) so a double click or retry records one piece of feedback, and spanId to attach it to one step instead of the whole run.

Record a score

Scores are typed values with a name, for example from an automated check or an LLM judge:

client.score(runId, "answer_quality", 0.92, { dataType: "numeric" });
client.score(runId, "followed_policy", true, { dataType: "boolean" });

client.score(run_id, "answer_quality", 0.92, data_type="numeric")
client.score(run_id, "followed_policy", True, data_type="boolean")

dataType is numeric, boolean, categorical or text.

Where do I get the run id?

Inside a traced function, call currentRunId() (TypeScript) or current_run_id() (Python) and return it with your response, so your UI can send it back with the feedback. Never trust a run id supplied by an end user without checking that the session owns it.

Use what you collect

Filter traces by low scores or negative feedback to find failures worth fixing, save them to a dataset, and run candidate prompts or models against them before you promote a change. See the evaluation engine and release gates.