Tool-Calling Reliability: Handling Failures, Retries, and Malformed Calls

At a glance

This guide is for engineers building or hardening tool-calling agents. It helps you turn an unreliable tool call into a bounded, observable operation so you can leave with a wrapper pattern that validates, retries, and fails safely without guessing.

The tool-call reliability framework

Use Validate → Bound → Classify → Recover as the decision path for every call:

1. Validate: check the model's proposed call — tool name, argument shape, and required fields — before anything executes.
2. Bound: attach a timeout and a maximum attempt budget before the call leaves the process.
3. Classify: decide whether the outcome is a success, a retryable failure, or a terminal failure.
4. Recover: retry within the bound, fall back to a safe alternative, or escalate to a human.

Decision rule: Validate before you execute, bound every attempt with a timeout and a retry ceiling, classify the result instead of assuming success, and recover through a path you chose in advance — not one improvised mid-incident.

This framework is deliberately narrower than the whole subject of agent reliability. It does not cover prompt design, tool selection policy, or multi-step planning; it covers the mechanics of a single tool call and the seams where it fails.

Map the tool-call lifecycle

A tool call looks like a single step in a transcript, but it is really four separate handoffs, and each one has its own failure mode. The model proposes an action. Your application parses and validates the proposed arguments. The tool executes against a real dependency — an API, a database, a browser, a payment processor. The result comes back and has to be interpreted before the model sees it again. Treating this as one atomic step is the single biggest source of undebuggable tool-calling incidents. A team that logs only "tool call succeeded" or "tool call failed" cannot tell whether the model asked for the wrong thing, the network dropped a request, or the tool returned a technically-valid response that meant nothing useful.

Why a confident-looking call still breaks

Large language models generate tool calls the same way they generate any other text: token by token, from a distribution shaped by training and by the current context. Nothing about the generation process guarantees that a tool name exists in your registry, that a required field is present, or that a numeric argument is inside a sane range. The model is not executing a plan against a schema; it is producing a string that looks like one. This is why validation has to happen in your application, never inside the assumption that "the model was told the schema, so it will comply." Even well-behaved models drift on edge cases: long conversations, unusual input, or a tool list that changed since the system prompt was last reviewed.

Name the four seams

Give each seam a name so your logs and your team can talk about the same failure the same way:

1. Proposal seam — the model chooses a tool and arguments.
2. Validation seam — your code checks the proposal before anything runs.
3. Execution seam — the tool runs against a real dependency under a timeout.
4. Interpretation seam — the result is classified before it re-enters the conversation.

Every failure mode in this guide maps to exactly one seam. That mapping is what makes an incident tractable: instead of "the agent is unreliable," you get "the validation seam rejected 4% of calls to search, and the rejection reason was always a missing query field."

Malformed and hallucinated calls

The proposal and validation seams fail in two distinct ways, and conflating them leads to the wrong fix. A malformed call has syntax or shape problems: the JSON does not parse, a required field is missing, or a value is the wrong type. A hallucinated call is syntactically fine but semantically wrong: the model invents a tool that does not exist, invents a parameter the tool never defined, or fabricates an identifier that looks plausible but was never returned by an earlier step.

Malformed JSON arguments

Most SDKs already guarantee syntactic JSON for structured tool-calling APIs, but do not assume that guarantee extends to your business rules. Parse defensively, and validate the parsed object against a real schema — not a set of scattered if checks scattered through the handler.

import { z } from "zod";

const searchArgsSchema = z.object({
  query: z.string().min(1).max(300),
  maxResults: z.number().int().min(1).max(20).default(5),
});

type SearchArgs = z.infer<typeof searchArgsSchema>;

function parseToolArguments(raw: string): SearchArgs {
  let candidate: unknown;
  try {
    candidate = JSON.parse(raw);
  } catch {
    throw new ToolValidationError("invalid_json", "Arguments were not valid JSON");
  }
  const result = searchArgsSchema.safeParse(candidate);
  if (!result.success) {
    throw new ToolValidationError("schema_mismatch", result.error.message);
  }
  return result.data;
}

Keep the raw string around when validation fails. It is the single most useful artifact for distinguishing a one-off model slip from a systematic prompt or schema regression — this is the same discipline covered in structured outputs for AI agents, applied specifically to the tool-call boundary rather than the whole decision object.

Hallucinated tool names and parameters

A hallucinated tool name is easy to catch — check it against a fixed registry and reject anything not in the list. Resist the temptation to fuzzy-match a near-miss name to the "closest" real tool. search_accounts silently routed to search_orders because the names are similar is a worse failure than a clean rejection, because it produces a plausible-looking wrong answer instead of a visible error.

Hallucinated parameters are subtler. A model can propose a syntactically valid call to a real tool with an argument value that was never grounded in anything — an account ID that looks right but was never returned by a prior step, a date computed from a wrong assumption about "today." Schema validation cannot catch this because the shape is correct. The mitigation is provenance tracking: when a tool result returns identifiers, tag them as "known-valid" in the working context, and validate that identifier arguments in the next call match a known-valid value rather than trusting free-form text the model generated from scratch.

Field note: Rejecting a hallucinated tool name with a clear error message that lists the available tools is often enough for the model to self-correct on the next turn. Silently dropping the call or retrying with the same broken proposal usually is not.

Timeouts and execution errors

Once a call clears validation, it has to run against something real, and real dependencies are slow, rate-limited, and occasionally down. The execution seam needs two separate controls: a timeout so a hung dependency cannot stall the whole agent turn, and an error classification so the system knows whether the failure is worth retrying.

Transient versus permanent failures

Treating every non-200 response as "retry the same request" is the most common tool-calling reliability bug in production agents. A 404 for a record that does not exist will still 404 on the fifth attempt; retrying just adds latency and cost while hiding a real problem from the model and the user.

Set a timeout per tool, not one global default

A single global timeout across every tool undersizes some calls and oversizes others. A cache lookup that takes longer than 300ms is already a signal something is wrong; a report-generation tool might legitimately need 20 seconds. Configure the timeout as tool metadata, not a constant buried in the retry loop.

const TOOL_TIMEOUTS_MS: Record<string, number> = {
  search: 4_000,
  get_record: 2_000,
  generate_report: 20_000,
  charge_payment: 8_000,
};

async function executeWithTimeout<T>(tool: string, fn: (signal: AbortSignal) => Promise<T>): Promise<T> {
  const controller = new AbortController();
  const timeoutMs = TOOL_TIMEOUTS_MS[tool] ?? 5_000;
  const timer = setTimeout(() => controller.abort(), timeoutMs);
  try {
    return await fn(controller.signal);
  } finally {
    clearTimeout(timer);
  }
}

Partial and ambiguous results

Not every failure looks like an error. A tool can return HTTP 200 with an empty list, a partially populated record, or a status field the model has never seen before. If your code treats "no exception was thrown" as "the call succeeded," these cases slip through as false positives — the agent proceeds as though it has good information when it does not.

Validate the shape of a successful result the same way you validate the arguments going in. Define what a complete, usable result looks like, and treat anything short of that as a failure to classify, not a success to forward.

const searchResultSchema = z.object({
  results: z.array(z.object({ id: z.string(), title: z.string() })),
  totalCount: z.number().int().nonnegative(),
});

function classifySearchResult(raw: unknown): "success" | "empty" | "invalid_shape" {
  const parsed = searchResultSchema.safeParse(raw);
  if (!parsed.success) return "invalid_shape";
  return parsed.data.results.length > 0 ? "success" : "empty";
}

An empty result is not always a failure — sometimes there genuinely are no matches — but it is a different case from a successful match, and the model's next step should be able to tell them apart. Collapsing "found nothing" and "the tool broke" into the same code path is a common source of agents that apologize for a real system fault as though it were a normal empty search.

Build the validation layer before execution

Validation belongs in a layer the model cannot bypass, checked before any side effect happens. Structure it as three separate checks rather than one bundled function, because each one fails for a different reason and needs a different response:

1. Shape validation — does the JSON match the schema?
2. Semantic validation — do the values make sense together (a start date before an end date, an amount within an allowed range)?
3. Authorization validation — is this caller, in this context, allowed to invoke this tool with these arguments?

Keep these checks in application code, not in the prompt. A system-prompt instruction such as "only call refund for orders under $50" is a suggestion the model can drift away from under an unusual conversation; a hard check in the tool handler is a boundary the model cannot talk its way past. This is the same separation covered in more depth in structured outputs for AI agents — the tool-calling boundary is one of the concrete places that separation pays for itself.

Retry strategy: backoff and idempotency

Retrying is not free, and retrying the wrong thing is actively dangerous. Before writing a retry loop, answer one question about the tool: is this action idempotent? A lookup, a search, or a dry-run computation can be retried freely — calling it twice has no side effect beyond doing the work twice. A charge, a message send, or a record mutation is not idempotent unless you explicitly make it so, and retrying it blindly after a timeout can produce a duplicate side effect that is worse than the original failure.

Give non-idempotent tools an idempotency key

Generate a stable key for the logical operation — not the HTTP request — and pass it through to the tool so the receiving system (or your own wrapper, if the tool doesn't support keys natively) can recognize a retry of the same intent and return the original result instead of repeating the action.

type ToolResult<T> = { status: "success"; data: T } | { status: "retryable_error"; reason: string } | { status: "terminal_error"; reason: string };

async function withRetry<T>(
  tool: string,
  idempotencyKey: string,
  attempt: (key: string, signal: AbortSignal) => Promise<ToolResult<T>>,
  { maxAttempts = 3, baseDelayMs = 250 } = {},
): Promise<ToolResult<T>> {
  let lastResult: ToolResult<T> | undefined;

  for (let attemptNumber = 1; attemptNumber <= maxAttempts; attemptNumber += 1) {
    const startedAt = Date.now();
    lastResult = await executeWithTimeout(tool, (signal) => attempt(idempotencyKey, signal)).catch(
      (error): ToolResult<T> => ({ status: "retryable_error", reason: error instanceof Error ? error.message : "unknown" }),
    );

    recordToolAttempt({ tool, idempotencyKey, attemptNumber, status: lastResult.status, elapsedMs: Date.now() - startedAt });

    if (lastResult.status === "success" || lastResult.status === "terminal_error") return lastResult;
    if (attemptNumber === maxAttempts) break;

    const backoffMs = baseDelayMs * 2 ** (attemptNumber - 1);
    const jitterMs = Math.random() * backoffMs * 0.3;
    await new Promise((resolve) => setTimeout(resolve, backoffMs + jitterMs));
  }

  return lastResult ?? { status: "terminal_error", reason: "no_attempts_made" };
}

This wrapper is illustrative — adapt recordToolAttempt, the error taxonomy, and the backoff curve to your own dependency behavior — but the shape holds generally: classify before deciding to retry, always pass the same idempotency key across attempts of the same logical call, and cap the attempt count instead of retrying indefinitely.

Decision rule: Retry a transient failure on an idempotent tool freely, within a bounded attempt count. Retry a transient failure on a non-idempotent tool only when you can prove the prior attempt did not already take effect. Never retry a permanent failure — fix the request or escalate instead.

Circuit breakers for flaky tools

A retry loop protects one call. A circuit breaker protects the whole system from a tool that has stopped working. When a dependency degrades, every agent turn that calls it pays the full timeout before failing, which multiplies latency across every concurrent user and can cascade into a broader outage. A circuit breaker tracks the failure rate for a tool over a rolling window and, once it crosses a threshold, stops sending calls to that tool for a cooldown period — failing fast with a known error instead of queuing up slow, doomed requests.

type CircuitState = "closed" | "open" | "half_open";

class ToolCircuitBreaker {
  private state: CircuitState = "closed";
  private failureCount = 0;
  private openedAt = 0;

  constructor(private readonly failureThreshold = 5, private readonly cooldownMs = 30_000) {}

  canAttempt(): boolean {
    if (this.state !== "open") return true;
    if (Date.now() - this.openedAt >= this.cooldownMs) {
      this.state = "half_open";
      return true;
    }
    return false;
  }

  onSuccess(): void {
    this.state = "closed";
    this.failureCount = 0;
  }

  onFailure(): void {
    this.failureCount += 1;
    if (this.state === "half_open" || this.failureCount >= this.failureThreshold) {
      this.state = "open";
      this.openedAt = Date.now();
    }
  }
}

Scope one breaker instance per tool, not one global breaker for the whole agent. A degraded send_email tool should not stop search from working. The half-open state matters as much as the open state: it lets exactly one probe call through after the cooldown, so the system can recover automatically instead of waiting for a human to flip a switch.

Fallback strategies when a tool is unavailable

Every tool call needs an answer to "what happens if this is simply not available right now" that does not involve the agent inventing an answer. Three fallback shapes cover most cases:

• Degrade to a cheaper or narrower tool. If a live-inventory lookup is down, fall back to a cached snapshot and label the response as possibly stale rather than failing the whole turn.
• Ask the user or a human reviewer. For consequential actions — anything involving money, irreversible state changes, or external communication — a bounded escalation is safer than a confident guess.
• Return an explicit, typed failure to the model. Give the model a clear "this tool is currently unavailable" result it can reason about and explain to the user, instead of a raw exception it will paper over with invented text.

The one fallback to avoid is silent degradation to guessing. If a tool cannot confirm a fact, do not let the model answer as though it had confirmed it — this is exactly the gap between a fluent response and a grounded one described in structured outputs for AI agents, and it applies just as directly when the missing evidence is a failed tool call rather than a missing document.

Try it: pick a failure mode

if (!schema.safeParse(args).success) return { status: 'terminal_error', reason: 'schema_mismatch' };

What to log and trace for debugging

A tool call that fails once in staging is a bug report. A tool call that fails once a week in production, for one customer, on one input class, is a mystery unless the trace carries enough context to reconstruct it. Record these fields on every attempt, success or failure:

Do not log the full prompt or unredacted arguments by default. A spike in schema_mismatch errors after a deploy is a useful, safe-to-share signal; the raw customer data inside a failed argument payload usually is not. Keep the redaction rule as strict for tool-call traces as for any other production log, and store the raw value only behind an access-controlled, short-retention path for authorized debugging. This connects directly to the broader practice in debugging AI agents in production: a trace is only useful if a person who was not in the room can read it and understand what happened.

A worked example

Consider a support agent that calls refund_order. A customer reports the refund never appeared. The trace shows three attempts: the first timed out at 8 seconds, the second returned an HTTP 409 ("refund already in progress"), and the third succeeded. Without the idempotency key and the classification field, this looks like three separate refund attempts and raises the question of a possible double refund. With them, it is a five-second story: the first attempt likely succeeded server-side after the client-side timeout, the second attempt correctly detected the in-flight duplicate and returned a safe conflict, and the third attempt's "success" was actually a confirmation of the original transaction rather than a second charge. That distinction — one real side effect versus three — is only recoverable if the idempotency key was generated once per logical action and threaded through every retry.

Test the failure paths, not just the happy path

A tool-calling test suite that only exercises successful calls will not catch the bugs that show up in production. Build fixtures for each seam explicitly: invalid JSON, a schema-valid but semantically wrong argument, an unknown tool name, a simulated timeout, each HTTP status class, and a 200 response with an empty or malformed body. Run these against the real validation and retry code, not a mocked-out version of it, so a change to the schema or the backoff curve gets caught before it reaches a user. AI agent testing covers building this kind of fixture set as part of a broader evaluation practice; tool-call failure fixtures are one of the highest-value places to start because they are deterministic and cheap to assert against.

Review the failure fixtures for coverage the same way you would review test coverage for any other critical path: does every branch in the retry classifier have at least one fixture that exercises it? Does the circuit breaker have a fixture that opens it and a fixture that confirms the half-open probe? A validation layer that has never seen a malformed call in CI will meet its first one in production.

Release checklist

• Every tool has a registered schema, and calls to unregistered tools are rejected, not fuzzy-matched.
• Every tool has an explicit timeout sized to its expected latency, not a shared global default.
• Non-idempotent tools require an idempotency key before a retry is allowed.
• Retries are bounded by attempt count and use backoff with jitter, not a fixed interval.
• A circuit breaker exists per tool, with a cooldown and a half-open probe.
• Fallback behavior is defined for at least the highest-traffic and highest-risk tools.
• Every attempt is logged with tool name, classification, attempt number, and elapsed time, with arguments redacted by default.
• Failure-path fixtures exist for malformed arguments, unknown tools, timeouts, each HTTP status class, and ambiguous results.

Decision rule: If a tool call cannot be validated, bounded, and classified, it is not ready to run unattended — put a human in the loop until it can be.

Frequently asked questions