Integrations / Haystack

Trace Haystack pipelines

To trace Haystack with Tracify, wrap the function that calls pipeline.run with trace_agent and open a step for the part you want to inspect. The run shows timing, inputs and outputs for each step.

Read the guideStart free

Your code

Haystack

Tracify

One run.

  • 01One trace per pipeline run
  • 02Retrieval as its own step
  • 03Generator output and timing
  • 04Errors that stop a pipeline

Setup

5 steps.

  1. 01

    Install

    Haystack and the Tracify SDK.

    bash
    pip install tracify-sdk haystack-ai
  2. 02

    Set your project key

    Create a project in Tracify, copy its API key and put it in the environment where your code runs. The SDK reads it from there, so nothing is hard-coded.

    bash
    export TRACIFY_API_KEY="tracify_sk_live_..."
  3. 03

    Wrap the pipeline call

    One function, one run. The step shows the whole pipeline as a single timed block with its input and the reply.

    python
    import tracify

@tracify.trace_agent(name="haystack-rag")
def answer(question: str) -> str:
    with tracify.span("pipeline", type="agent", input=question) as step:
        result = pipeline.run({
            "retriever": {"query": question},
            "prompt_builder": {"question": question},
        })
        reply = result["llm"]["replies"][0]
        step.set_output(reply)
        return reply
  4. 04

    Split out retrieval

    To see retrieval separately from generation, run the retriever on its own inside its own step.

    python
    @tracify.trace_agent(name="haystack-split")
def answer(question: str) -> str:
    with tracify.span("retrieve", type="retrieval", input=question) as step:
        docs = retriever.run(query=question)["documents"]
        step.set_output([d.content[:200] for d in docs])
    with tracify.span("generate", type="llm", input=question) as step:
        out = generator.run(prompt=build_prompt(question, docs))
        step.set_output(out["replies"][0])
        return out["replies"][0]
  5. 05

    Run it once and look

    Run your code one time, then open the Tracing page of your project. Your pipeline shows up as a run with its steps in order, plus the tokens, time and any error for each call.

Good to know

Per-component spans are not automatic

The SDK does not hook into every Haystack component by itself. You decide which parts are worth a step, and each step you add shows up in the run.

Full details in the documentation. All integrations are listed on the integrations page.

Can I see which documents the retriever returned?
Yes. Set the step's output to the documents, or a short preview of each, and open it in the trace.
Does it work with Haystack agents that use tools?
Yes. Wrap the agent call with trace_agent and open a step in each tool function so every tool call appears in order.
Is the model call itself recorded?
If your generator uses an OpenAI-style client that you create and wrap, yes. Otherwise record it with a step and set the tokens from the reply metadata.

Send your first trace.

Free to start.

Start freeCompare tools