FIELD NOTE / ARCHITECTURE NOTE
Everything Around the Model: The Real Engineering of Production AI Agents
Six weeks after a logistics team shipped a freight-exception agent, it began rerouting shipments from stale memory, retrying carrier calls until bookings duplicated, and planning against warehouse state that no longer existed. Nothing crashed. The agent kept running and kept being wrong. This note follows that incident to the boundary where production agents are actually built: planning, reasoning, execution, tools, memory, state, and the controls around them.
The LLM is infrastructure, not the agent
The freight agent looked excellent in a controlled test because the model was given clean state and forgiving tools. Production added delayed carrier responses, partial warehouse feeds, and concurrent operators. The model was a reasoning substrate, like a CPU: useful, fast, and indifferent to whether the software around it was reliable. Calling the LLM the agent hid ownership. The planner, state store, executor, tool registry, memory policies, and release controls made the decisions observable or opaque. In logistics, 6.2% of exception runs touched stale state during week six, while model quality on the original test set remained unchanged.
Planning is a changing graph
A planner decomposes a goal into a directed acyclic graph with dependencies, preconditions, and expected outputs. One-shot planning generated all eight freight actions before the first carrier observation; step-one accuracy was 96%, step-five 78%, and step-eight 61% in a financial workflow automation deployment. Iterative replanning repaired some drift, but the trigger was the hard part: abandon a plan when a precondition changes, not merely when a tool throws. The freight agent needed a replan when a carrier reservation changed, even though every API returned 200.
Reasoning is a stateful inference loop
Planning decides what should happen; reasoning decides what to do next from the state currently known. The useful primitive is ReAct: reason, act, observe, reason. In legal review, reasoning accuracy was 94% at step three when the context window was 28% full, but 67% at step eleven when it was 86% full. Load-bearing observations were truncated, recent guesses dominated older evidence, and contradictions accumulated. The freight agent’s route choice was locally coherent because the reasoning engine could no longer see the warehouse closure that caused the exception.
The execution layer meets the world
Execution is not a wrapper around a function call. It enforces timeouts, exponential backoff, idempotency, transient-versus-permanent classification, and result validation before returning an observation. The freight agent had none of these guarantees: a timeout was retried without an idempotency key and the carrier processed both bookings. In a payments workflow, adding bounded retries and response validation increased p95 latency by 18 ms and reduced duplicate settlement attempts from 2.6% to 0.08%. The reasoning layer should receive a typed result—success, retryable failure, permanent failure, or unknown—not a raw exception string.
Tools are the capability boundary
A production tool needs an input schema, output schema, latency contract, failure specification, authorization scope, and trace surface. An implicit callable cannot tell an agent whether an empty result means no records or a broken connection. A clinical-trial extraction agent received empty database results during connection-pool exhaustion and concluded that no matching trials existed; patient routing changed for 41 cases before the issue was detected after 27 minutes. Tool output poisoning is a reasoning failure only after the interface has allowed malformed data to look valid.
Memory has four distinct failure modes
Short-term memory holds current task context and fails by overflow; in e-commerce order resolution, truncation removed a shipping constraint in 11% of long traces. Long-term memory persists experience and fails by staleness; an insurance agent used a carrier rule retired 19 days earlier. Semantic memory stores retrievable facts and fails when query and storage representations miss each other; a pharma assistant’s recall fell from 0.91 to 0.68 after a terminology update. User-preference memory fails by drift; a banking assistant stored email as preferred, while a newer session selected Slack, producing conflicting notifications in 7% of cases. Each layer needs its own retention, retrieval, and invalidation policy.
Persists for this run · failure mode: overflow
Persists across runs · failure mode: staleness or drift
Persists across runs · failure mode: staleness or drift
Persists across runs · failure mode: staleness or drift
State is not memory
State is what the current run needs to reason correctly: the plan DAG, step number, last tool result, open observations, permissions, and pending side effects. Memory persists information for future retrieval: a shipper preference, a recurring carrier delay, or a warehouse capacity fact with a freshness timestamp. The freight implementation wrote retry payloads and temporary route candidates into long-term memory, while failing to copy the current carrier reservation into state. Retrieval returned yesterday’s preference and the planner treated it as today’s truth. The result was 14 duplicate bookings and a 3.4-hour investigation. The boundary should be explicit, typed, and reviewed like an API.
Six production properties
Guardrails validate every intermediate output; without them, a malformed object survives until a final answer. Observability records plans, prompts, tool calls, memory reads and writes, and state deltas; without it, the freight team saw only a successful task count. Retries need bounded budgets and idempotency; otherwise a transient timeout becomes a duplicate side effect. Timeouts exist at tool, step, and task levels, with graceful degradation at each boundary. Idempotency makes resubmission safe. Security sandboxes tools, sanitizes inputs and outputs, and applies least privilege. A regulated document workflow added these controls at 7% compute cost and cut unreplayable incidents by 79%.
Not production-ready: the agent has no reliable containment.
What orchestration frameworks change
LangGraph-like systems provide an explicit state machine, node-level error handling, checkpoints, and human-in-the-loop insertion points. They do not decide memory freshness, enforce every tool contract, or measure reasoning degradation. With explicit transitions, the freight run would have stopped after carrier availability changed, replayed from the last valid checkpoint, and shown the exact state delta. With implicit accumulation, the same run appended observations until the route plan looked internally consistent. Checkpointing is the structural insight: it supports debugging, recovery, and evaluation without pretending the model is deterministic.
Observability must be designed first
A production trace needs the planning DAG, every reasoning input context, every tool input and output, every memory query and result IDs, every memory write and retention policy, and every replan trigger and delta. Redact sensitive payloads but preserve hashes, versions, and provenance. In logistics, adding traces after the incident recovered only 61% of the missing fields because the original agent never emitted state diffs. A minimum viable surface is a run ID, state schema version, node span, tool contract version, model and prompt version, latency, retry count, and outcome classification. Anything less turns a failure into a story instead of a replay.
Plan → Reason → Execute → Observe → Replan
This loop is the reliability unit. Measure planning quality against a task graph, reasoning quality as context grows, execution reliability separately from tool reliability, and observation integration separately from both. Re-instrumenting the freight agent revealed 92% planner validity, 88% reasoning agreement at normal context, 99.2% tool availability, but only 83% observation integration because stale memory writes were silently accepted. The aggregate task score had hidden the boundary. Once each traversal had a typed outcome, the team could fix the memory write path without changing the model.
A production-ready agent is diagnosable
Can the team show the current state contract without opening source code? Can it explain when a plan is abandoned, which evidence authorized the next action, and whether a retry can safely repeat a side effect? Can it replay from a checkpoint, identify stale memory, and separate a tool outage from a reasoning error? Where does a human approve an irreversible action? Which metrics alert before users do? The re-architected freight agent added a firewall executor, freshness-aware memory, bounded retries, circuit breakers, and checkpoint replay. Correct exception resolution rose from 71% to 94%, duplicate bookings fell 2.6% to 0.08%, and mean detection time fell from 27 minutes to 54 seconds. That is the product: not a clever prompt, but a system that can be trusted when the world changes.
Share a thought
Comments appear immediately. Email is optional and never shown.
No comments yet. Be the first to share a thought.