Integrations / Ollama

Trace local Ollama models

To trace Ollama with Tracify, use the OpenAI client pointed at Ollama's local /v1 endpoint and wrap it with the Tracify SDK. Each local call is recorded with the model, tokens and latency.

Read the guideStart free

Your code

Ollama

Tracify

One run.

  • 01Local model name and tag
  • 02Token counts reported by Ollama
  • 03Latency per call, including slow cold starts
  • 04Calls grouped into the agent run

Setup

6 steps.

  1. 01

    Make sure Ollama is running

    Start Ollama and pull a model once. The compatible API listens on port 11434.

    bash
    ollama pull llama3.2
  2. 02

    Install

    The OpenAI library talks to Ollama, so no Ollama-specific package is needed.

    bash
    pip install tracify-sdk openai
  3. 03

    Set your project key

    Create a project in Tracify, copy its API key and put it in the environment where your code runs. The SDK reads it from there, so nothing is hard-coded.

    bash
    export TRACIFY_API_KEY="tracify_sk_live_..."
  4. 04

    Wrap the client

    Ollama ignores the API key but the library wants one, so any text works.

    python
    import tracify
from openai import OpenAI

ollama = tracify.wrap_openai(OpenAI(
    base_url="http://localhost:11434/v1",
    api_key="ollama",
))

reply = ollama.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Hello"}],
)
  5. 05

    Or in TypeScript

    Same endpoint, same wrapper.

    typescript
    import OpenAI from "openai";
import { wrapOpenAI } from "tracify-sdk";

const ollama = wrapOpenAI(
  new OpenAI({ baseURL: "http://localhost:11434/v1", apiKey: "ollama" }),
);

const reply = await ollama.chat.completions.create({
  model: "llama3.2",
  messages: [{ role: "user", content: "Hello" }],
});
  6. 06

    Run it once and look

    Run your code one time, then open the Tracing page of your project. Your local call shows up as a run with its steps in order, plus the tokens, time and any error for each call.

Good to know

Shown with the OpenAI label

The wrapper reads the OpenAI call format, so calls are labeled as OpenAI-style in the trace. The model name you send is always the one shown, so you can still tell models apart.

Good to know

Your prompts leave your machine

The model runs locally, but Tracify is a hosted service, so the prompts and answers you record are sent to it. To keep content out, turn off input and output capture on the wrapper, or use the redaction options in the SDK.

Good to know

Local models cost nothing per call

Tracify only shows a dollar cost when you pass your own rates. For local models you can leave it out, or put in an estimate of your hardware cost per token.

Full details in the documentation. All integrations are listed on the integrations page.

Does tracing Ollama change my requests or responses?
No. The wrapper passes your call straight through and records a copy. Your request, the response and any error reach your code unchanged.
Can I use Ollama's own Python library instead?
Not through the wrapper. The wrapper covers OpenAI-style clients. With Ollama's own library, record the call as a step and copy the token counts from the response.
Why is my first call so slow?
Ollama loads the model into memory on the first call. In the trace you will see that first call take much longer than the rest.

Send your first trace.

Free to start.

Start freeCompare tools