AI agents are expensive because they combine multiple cost drivers: LLM tokens (for reasoning at each step), tool calls (each tool call may consume API credits), and compute (the agent loop runs until completion). A single agent request may consume 5-20x more tokens than a single LLM call. This makes agent cost optimisation critical for production viability.
Tool Call Optimisation
Each tool call costs time and money. Optimise with: (1) Tool result caching: cache tool results for identical inputs. If the agent calls the same tool with the same parameters, return the cached result. (2) Tool call batching: if the agent needs to call the same tool multiple times, batch the calls into a single request. (3) Tool selection optimisation: train the agent to choose the cheapest appropriate tool. A database query is cheaper than a web search. A cache lookup is cheaper than a database query.
LLM Token Optimisation
Agents consume tokens at every step of the loop. Optimise with: (1) Conversation summarisation: compress long conversations into summaries. Reduces context token count 40-60%. (2) System prompt optimisation: shorter, more concise system prompts. (3) Response streaming: start generating tokens immediately, stop when the agent has enough information. (4) Model routing: use smaller models for simple agent steps (classification, extraction) and larger models for complex steps (planning, reasoning).
Pro Tip
Conversation summarisation is the highest-impact agent optimisation. Long conversations are the primary driver of agent costs — summarise aggressively.
Loop Optimisation
The agent loop runs until the goal is achieved. Optimise with: (1) Step limits: maximum steps per task prevents runaway loops. (2) Early termination: if the agent determines it cannot achieve the goal, stop early instead of continuing to generate tokens. (3) Confidence thresholds: if the agent is confident in its answer, skip additional tool calls. (4) Caching at the loop level: if the agent has handled a similar task before, replay the tool call sequence instead of re-reasoning.
Conclusion
Agent costs are optimisable at three levels: tool calls (caching, batching, selection), LLM tokens (summarisation, compression, routing), and loop (step limits, early termination, caching). Conversation summarisation is the highest-impact optimisation — start there.
Key Takeaways
- Agent costs: LLM tokens (reasoning) + tool calls (API credits) + compute (loop execution)
- Tool optimisation: result caching, call batching, cheaper tool selection
- Token optimisation: conversation summarisation (highest impact), prompt compression, model routing
- Loop optimisation: step limits, early termination, confidence thresholds, loop-level caching
- Conversation summarisation reduces context tokens 40-60% — start there