Evaluating a chatbot is straightforward: did it give the right answer? Evaluating an agent is complex: did it choose the right tools? Did it call them with the right parameters? Did it plan the right sequence? Did it produce the right output? Did it do all of this within acceptable latency and cost? Agent evaluation requires a multi-dimensional framework that measures quality at every step, not just the final output.
The Five Evaluation Dimensions
Agent evaluation has five dimensions: (1) Task completion — did the agent achieve the goal? (2) Tool selection — did it choose the right tools? (3) Parameter accuracy — did it pass the right arguments to each tool? (4) Planning efficiency — did it execute the minimum steps necessary? (5) Safety — did it respect all guardrails and permission boundaries? Each dimension is independently measurable and requires different evaluation techniques.
Building Evaluation Datasets
Create a test dataset of 50-100 representative tasks with expected outcomes. For each task, define: the expected tool call sequence, the expected parameters, the expected final output, and any safety constraints that should be triggered. This dataset is your regression suite — run it after every change to the agent's prompts, tools, or planning logic. Maintain the dataset as a living document: add new test cases when you discover edge cases in production.
Pro Tip
The most valuable test cases are the ones that failed in production. Add every production failure to your evaluation dataset.
Automated vs Human Evaluation
Automated evaluation catches: wrong tool calls, incorrect parameters, guardrail violations, and latency/cost threshold breaches. Human evaluation catches: subtle quality issues, inappropriate tone, missing context, and tasks that technically succeeded but produced poor outcomes. Use automated evaluation for every deployment (continuous regression testing). Use human evaluation for periodic quality audits (weekly or monthly sample review). The ratio is typically 80% automated, 20% human.
Regression Detection
The most dangerous evaluation gap is regression: a change that improves one task but degrades another. Implement a comparison pipeline that runs the full evaluation dataset before and after every change, reports any degradation, and blocks deployment if degradation exceeds a threshold. This prevents the common pattern: 'we fixed the billing agent and broke the support agent.' Every prompt change, tool update, or model upgrade should go through this pipeline.
Conclusion
Agent evaluation is multi-dimensional and ongoing. Build a comprehensive test dataset, automate regression testing, supplement with human evaluation, and treat every production failure as a new test case. The evaluation framework is the quality foundation that makes continuous improvement possible.
Key Takeaways
- Evaluate five dimensions: task completion, tool selection, parameter accuracy, planning efficiency, safety
- Build a 50-100 task test dataset with expected outcomes — maintain it as a living document
- Automated evaluation for every deployment; human evaluation for periodic quality audits
- Regression detection: compare full evaluation dataset before and after every change
- Every production failure becomes a new test case — the evaluation dataset grows over time