The happy path proves very little
Most agent demos begin with a cooperative customer and end before anything ambiguous happens. Production begins where that demo stops: partial information, conflicting policies, frustration, interruptions, and requests the agent should refuse.
Agor turns those moments into a reusable simulation set before a release can be marked live.
Test behavior, not phrasing
The judge should care whether the job was completed, policy was followed, uncertainty was surfaced, and the handoff preserved context. It should not reward one preferred sentence over another equally good sentence.
The workspace reports both a first-attempt rate and a repeated-attempt rate. An agent that succeeds once and fails the next three times is not reliable enough for a customer moment that matters.
A failed test is product data
Failures are not buried. They become a queue: missing knowledge, unclear rules, tool errors, and supervisor interventions. The next release should be able to point to the exact failures it closes.