How do you evaluate the quality of AI agents?

Agents are harder than single LLM calls because they take multi-step trajectories with tool use, so evaluate both the outcome and the path. Outcome: did it complete the task correctly? Define task-success criteria on a set of representative goals and score end-state, ideally in a sandboxed environment you can check programmatically. Trajectory: was the reasoning sound, did it call the right tools with valid arguments, avoid unnecessary or unsafe steps, and recover from errors? Also track efficiency — step count, tokens, latency, cost — since a correct-but-wasteful agent is a problem. Observability is essential: log every step, tool call, and decision so failures are debuggable. Combine automated task-success checks with human or LLM-judge review of trajectories, and red team for unsafe tool actions.