Product / Runtime control
Limits you chose on purpose.
Set per-run and daily cost ceilings, a latency budget, an ordered fallback chain and a retry policy. Watch what the policy would do in observe mode, then enforce it.
- Mode
- Observe first
- Ceilings
- Per run, daily
- Fallbacks
- Ordered
Instrument it
Run the model call through a policy.
orchestrate wraps a call with your fallback chain, retry policy and ceilings. Start in observe mode and switch to enforce when the numbers look right.
$ npm install tracify-sdk
const reply = await client.orchestrate({
model: "large-model",
input: question,
call: (model, signal) => callModel(model, question, { signal }),
policy: {
enforcementMode: "observe",
maxCostPerRun: 0.25,
fallbackChain: ["fast-model"],
retryPolicy: { maxAttempts: 2, backoffMs: 1000, backoffMultiplier: 2, retryableErrors: ["timeout", "rate_limit"] },
latencyBudgetMs: 8000,
},
});Policy, not guesses
Watch the policy before it bites.
Use traces and cost analysis to understand ordinary and worst-case execution before selecting limits.
Record what would have been blocked or rerouted without changing live behavior.
Enable enforcement deliberately and watch fallback cost, retries, latency, and failures after release.
Baseline
$0.42 max
Fallback chain
large model → fast model → stop
Look at ordinary and worst-case runs before you pick a number.
Make limits and recovery a decision, not an accident.
Observe or enforce
Evaluate a policy without blocking runs, then move to enforcement when its thresholds are understood.
Cost ceilings
Set positive per-run and daily limits that can be checked by the orchestration path.
Fallback chains
Define an ordered list of model identifiers for recovery rather than allowing implicit provider behavior.
Retry and latency policy
Control attempt count, initial backoff, multiplier, retryable error classes, and latency budget.
Who it is for
Guardrails that live outside your app code.
Platform teams that need consistent guardrails across agent runs without hiding the policy in application code.
Questions people ask.
What is runtime control for AI agents?
Runtime control lets teams set explicit execution policy, such as cost ceilings, latency budgets, model fallback chains, and retry behavior, instead of hardcoding those limits into application code.
What is observe mode versus enforce mode?
Observe mode records what a policy would have blocked or rerouted without changing live behavior. Enforce mode actively applies the policy once its thresholds are validated.
How do I set a cost ceiling for an AI agent?
Define a positive per-run and daily spend limit based on baseline traces, run it in observe mode first, then enable enforcement once the limit reflects real production behavior.
What is a model fallback chain?
A fallback chain is an ordered list of model identifiers used for recovery when a primary model call fails or times out, rather than relying on implicit provider behavior.
How do I set a retry policy for LLM calls?
Configure attempt count, initial backoff, backoff multiplier, and which error classes are retryable, then monitor retry and fallback cost after enabling the policy.
See your own runs like this.