Integrations / LiteLLM

Trace LiteLLM calls and proxy traffic

To trace LiteLLM with Tracify, point an OpenAI client at your LiteLLM proxy and wrap it with the Tracify SDK. If you call litellm.completion directly in Python, record the call as a step with its token usage.

Read the guideStart free

Your code

LiteLLM

Tracify

One run.

  • 01The model name you send through LiteLLM
  • 02Token usage from the response
  • 03Latency including provider fallbacks
  • 04Calls grouped into the agent run

Setup

5 steps.

  1. 01

    Install

    For the proxy route you only need the OpenAI library.

    bash
    pip install tracify-sdk openai
  2. 02

    Set your project key

    Create a project in Tracify, copy its API key and put it in the environment where your code runs. The SDK reads it from there, so nothing is hard-coded.

    bash
    export TRACIFY_API_KEY="tracify_sk_live_..."
  3. 03

    Option A: through the LiteLLM proxy

    The LiteLLM proxy speaks the OpenAI format. Point your client at it and wrap that client. Every provider behind the proxy is traced the same way.

    python
    import tracify
from openai import OpenAI

llm = tracify.wrap_openai(OpenAI(
    base_url="http://localhost:4000",
    api_key="your-litellm-key",
))

reply = llm.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello"}],
)
  4. 04

    Option B: litellm.completion in your code

    The litellm function is not wrapped by the SDK, so record it as a step. LiteLLM returns usage in the OpenAI shape.

    python
    import litellm
import tracify

def ask(model: str, prompt: str) -> str:
    messages = [{"role": "user", "content": prompt}]
    with tracify.span("litellm", type="llm", input=messages) as step:
        response = litellm.completion(model=model, messages=messages)
        text = response.choices[0].message.content
        step.set(
            model=model,
            input_tokens=response.usage.prompt_tokens,
            output_tokens=response.usage.completion_tokens,
        )
        step.set_output(text)
        return text
  5. 05

    Run it once and look

    Run your code one time, then open the Tracing page of your project. Your LiteLLM call shows up as a run with its steps in order, plus the tokens, time and any error for each call.

Good to know

Shown with the OpenAI label

The wrapper reads the OpenAI call format, so calls are labeled as OpenAI-style in the trace. The model name you send is always the one shown, so you can still tell models apart.

Good to know

Fallbacks and the model name

Tracify records the model name you asked for. If LiteLLM falls back to another provider, the extra time shows up in the latency of that one call.

Good to know

About cost

Tracify records the tokens the provider returns. A dollar cost is shown when you give the wrapper your own rates with calculate_cost (Python) or calculateCost (TypeScript), so the number always matches your real price plan.

Full details in the documentation. All integrations are listed on the integrations page.

Does tracing LiteLLM change my requests or responses?
No. The wrapper passes your call straight through and records a copy. Your request, the response and any error reach your code unchanged.
LiteLLM already logs calls. Why add Tracify?
LiteLLM sees single model calls. Tracify puts those calls inside the agent run that made them, next to your tool calls and decisions.
Does it work with the proxy in a team setup?
Yes. Each service wraps its own client, so each one sends its own traces under its own project key.

Send your first trace.

Free to start.

Start freeCompare tools