Making AI Agents Reliable in Production: Retries, Idempotency and Human-in-the-Loop

Getting an AI agent to work in a demo takes an afternoon. Getting it to work reliably for thousands of real users, with flaky APIs, ambiguous requests, adversarial inputs and costs that have to make sense, is an engineering discipline. This guide collects the practices that close that gap.

#Key takeaways

• Treat the LLM as an unreliable component inside a reliable system. Wrap it with validation, limits and fallbacks.
• Idempotent tools make retries safe. This is the most important property for write actions.
• Use durable execution (persisted state) for long-running agent tasks so crashes don't lose progress or repeat side effects.
• Add human approval at consequential or low-confidence points.
• Trace everything and test with scenario evals before every release.

#Why agents fail in production

1. Model errors: wrong tool, wrong arguments, misread results, premature "done".
2. Tool errors: timeouts, rate limits, 500s, schema changes in downstream APIs.
3. Loops: repeating the same failing call, or oscillating between two actions.
4. Context problems: huge tool outputs crowd out instructions; long sessions drift off-task.
5. Ambiguity: the user's request was unclear, and the agent guessed instead of asking.
6. Adversarial input: prompt injection via user messages or retrieved content (AI guardrails).

Each needs a specific defense.

#1. Hard limits on everything

pythonLIMITS = {
    "max_steps": 12,
    "max_tool_calls_per_tool": 4,
    "max_total_tokens": 150_000,
    "max_wall_time_s": 120,
    "max_cost_usd": 0.50,
}

When a limit is hit, stop cleanly: summarize progress, save state and escalate to a human or return a graceful partial result. Never let an agent run unbounded.

#2. Idempotent tools

If a tool call times out, did the refund happen or not? Without idempotency, retrying might refund twice. Give every write operation an idempotency key derived from the task:

pythondef issue_refund(ctx, order_id: str, amount: Decimal):
    key = f"refund:{ctx.task_id}:{order_id}"
    existing = refunds.find_by_key(key)
    if existing:
        return {"status": "already_refunded", "refund_id": existing.id}
    refund = payments.refund(order_id, amount, idempotency_key=key)
    refunds.save(key=key, refund_id=refund.id)
    return {"status": "refunded", "refund_id": refund.id}

Many payment and messaging APIs accept idempotency keys natively. Use them. The same principle powers reliable message queue consumers.

#3. Retries at the right layer

• Transient tool failures (timeouts, 429, 503): retry inside the tool with exponential backoff and jitter, a few times, before returning an error to the model.
• Model API failures: retry with backoff; consider falling back to an alternate model.
• Invalid model output (bad JSON, schema violations): return the validation error to the model once or twice so it can self-correct, then fail.
• Semantic failures (wrong answer): no retry loop fixes this. You need better prompts, tools or evals.

#4. Validate every action

The model proposes; code disposes. Before executing:

• Validate arguments against schemas and business rules (amounts, allowed recipients, date ranges).
• Check authorization against the user's permissions, not the agent's.
• Verify preconditions (the order is delivered, the account is active).

After executing, verify postconditions where cheap. For example, re-fetch the record and confirm the update.

#5. Durable execution for long tasks

Agents that run for minutes or hours, or wait on humans, need state that survives crashes and deploys:

• Persist each step's input, output and status to a database.
• On restart, resume from the last completed step instead of starting over.
• Combined with idempotent tools, this gives effectively exactly-once side effects.

Workflow engines with durable execution semantics, or a simple step table plus a job queue, both work. The key is that the transcript and task state live outside process memory.

#6. Human-in-the-loop, designed deliberately

Humans are a feature, not a failure mode. Common checkpoints:

Make approvals fast. Show the proposed action, the evidence and a one-click approve, edit or reject. Log reviewer decisions; they're gold for evals and fine-tuning.

#7. Context hygiene

• Compact tool results: return only the fields the agent needs.
• Summarize older steps in long runs, keeping key facts and IDs in structured state.
• Pin the goal and constraints so they stay visible, rather than getting buried under 40 tool results.
• Separate untrusted content clearly from instructions.

#8. Loop detection

Detect and break:

• The same tool with the same arguments called repeatedly.
• Alternating between two states without progress.
• Steps increasing without any new information.

On detection, inject a message ("You've called search with the same query 3 times; try a different approach or conclude"), or stop and escalate.

#9. Observability

For every agent run, capture a trace:

• Run ID, user, model versions, prompt version.
• Each step: model input size, output, tool calls, tool results, latency and tokens.
• Final outcome, cost and any human interventions.

Dashboards to build: success rate by task type, escalation reasons, average steps and cost per task, tool error rates, and latency percentiles. Set alerts on spikes in errors, costs or escalations. Standard practices from observability apply directly.

#10. Scenario evals before every release

Build a suite of realistic scenarios with mocked tools, and assert on:

• Outcome: was the task completed correctly?
• Trajectory: were the right tools used, and were no forbidden actions attempted?
• Efficiency: steps, tokens and cost within budget.
• Safety: injection attempts and out-of-scope requests handled correctly.

Run each scenario several times to measure pass rates, since behavior is stochastic. Add every production incident as a new scenario. See how to evaluate LLM applications.

#11. Graceful degradation

When things go wrong, fail usefully:

• If the model API is down, fall back to a simpler flow, queue the task, or route to humans.
• If a tool is down, tell the user honestly and offer an alternative.
• Always return something actionable, never a stack trace or silence.

#Production readiness checklist

• Step, token, time and cost limits enforced
• All write tools idempotent, with keys tied to the task
• Retries with backoff inside tools; model fallbacks configured
• Argument validation and authorization in code
• Durable state for long-running tasks
• Human approval for consequential actions
• Loop detection
• Full tracing and dashboards, with alerts
• Scenario eval suite in CI with pass-rate thresholds
• Graceful degradation paths documented and tested

#FAQ

#How reliable can AI agents get?

For narrow, well-tooled tasks with good guardrails, agents can achieve high success rates, and the remaining failures get caught by validation and escalation. Broad, open-ended agents are much less predictable. Scope is the biggest reliability lever.

#Should I use a framework for durable agents?

If your agents run long tasks, wait on humans or perform many side effects, a workflow engine or framework with durable execution saves a lot of work. For short, synchronous agents, a simple loop with good logging is often enough.

#How do I test non-deterministic agents?

Mock tools for repeatability, run each scenario multiple times, and track pass rates rather than single pass/fail results. Assert on outcomes and forbidden actions rather than exact wording.

#What's the most common production failure?

In our experience, poor tool design: vague descriptions, bloated outputs and unhelpful errors. Improving tools often raises reliability more than switching models. See building AI agents with tool calling.