All documentation

Observe / guide 09

What is in a trace

The fields on every run and span, and how parents, runs and traces fit together.

Copy or share.

What is in a trace

A trace is the record of one agent run. It is made of spans: one root span for the run, and a child span for every model call, tool call, retrieval or decision inside it. Knowing the fields makes the trace viewer, filters and costs easier to read.

Runs, traces and spans

  • A run is one execution of your agent. Every span in the run shares a runId.
  • A span is one step. It has its own spanId and, unless it is the root, a parentSpanId pointing at the step that started it.
  • The root span closes the run. It is sent as run_end when the function returns, or as error when it throws, which marks the run completed or failed.

Parent and run links come from async context, so you rarely set them yourself. Two runs executing at the same time never mix their steps.

Span fields

Every span carries these fields. Only spanType is required.

  • Identity: spanId, runId, parentSpanId, traceName, spanType, createdAt.
  • Content: input, output. Both are serialized, redacted of obvious secrets, and capped at 10,000 characters.
  • Model work: modelId, inputTokens, outputTokens, costUsd, ttftMs (time to first token), retryCount.
  • Tool work: toolName.
  • Timing: latencyMs.
  • Failure: errorType, errorMessage.
  • Streaming: isStreamChunk, streamSequence, streamFinal.
  • Context: sessionId, endUserId, environment, release, deploymentId, tags.
  • Prompts: promptVersionId, which links a model call to the deployed prompt version it used.
  • Anything else: metadata, a free-form object.

Span types

The type you pass to span or observe is one of agent, llm, tool, retrieval or span. Use agent for a block that makes decisions, llm for a model call, tool for something outside the model, and retrieval for a lookup that feeds the model. The type decides the icon, the filters, and how the trace graph groups repeated work.

Reading a trace in the dashboard

  1. Start at the root and confirm the session, user, environment and release.
  2. Follow the waterfall to the slowest or failed child span.
  3. Open the input and output at the step where behavior changed.
  4. Compare the prompt version, model and tool arguments against a known-good run.

See Trace and inspect agent runs for the dashboard workflow, and Instrument tool calls for what to record around tools.