Building an agent that works in a demo is easy. Building an agent that works reliably in production — handling edge cases, respecting boundaries, explaining its decisions, and failing gracefully — requires a fundamentally different architectural approach. The difference between a demo agent and a production agent is not capability; it is guardrails.
The Agent Architecture Stack
A production agent has six architectural layers: (1) Interface layer — receives user input and returns outputs. (2) Planning layer — decomposes goals into sub-tasks. (3) Tool layer — executes actions via tool calls. (4) Memory layer — stores and retrieves context. (5) Guardrail layer — validates inputs, constrains actions, and catches errors. (6) Observability layer — logs every decision, action, and outcome. Most demo agents have layers 1-4. Production agents need layers 5-6.
Tool Sandboxing
Every tool call should execute in a sandbox with: permission boundaries (which tools the agent can access), rate limits (how many calls per minute), timeout constraints (maximum execution time), output validation (checking the tool's response for correctness and safety), and rollback capabilities (undoing actions when needed). The sandbox prevents the agent from taking unintended actions, consuming excessive resources, or causing cascading failures.
Pro Tip
Implement tool confirmation for high-risk actions: 'The agent wants to send an email to 500 customers. Confirm?' Human confirmation gates are not a failure of automation — they are a feature of safety.
Observability for Agents
Agent observability is fundamentally different from application observability. Application observability tracks: request rate, error rate, latency. Agent observability tracks: decision quality (what did the agent decide at each step?), tool usage (which tools did it call, with what inputs?), reasoning chain (why did it choose this action over that one?), outcome quality (did the final output meet the user's goal?), and resource consumption (how many LLM tokens, how many tool calls, how much time?). Without this visibility, debugging agent failures is nearly impossible.
The Guardrail Pattern
Implement guardrails at three levels: input guardrails (validate and sanitise user input — reject prompt injection attempts, enforce input length limits), action guardrails (validate each tool call before execution — check permissions, enforce rate limits, require confirmation for high-risk actions), and output guardrails (validate the agent's final output — check for hallucinations, verify factual claims, ensure the output matches the expected format). Each level catches different failure modes. All three are necessary.
Conclusion
Production agent architecture is about managing the agent's power, not increasing it. The guardrail layer, observability layer, and tool sandboxing are what separate a production agent from a demo. Build the guardrails first, then the capabilities.
Key Takeaways
- Production agents need six layers: interface, planning, tools, memory, guardrails, observability
- Tool sandboxing: permission boundaries, rate limits, timeouts, output validation, rollback
- Agent observability tracks decisions, tool usage, reasoning chains, and outcomes — not just uptime
- Guardrails at three levels: input, action, and output — all three are necessary
- Build guardrails first, then capabilities — safety is an architecture decision