Evaluate the Handoff, Not Just the Answer

An AI agent can produce a correct result and still make the wrong decision about when to act alone or ask for approval. I’d evaluate both.

For each workflow, test four outcomes: completes within its permission, escalates when approval is required, pauses when key information is missing, and explains what needs review. Include cases where the right move is to stop—not just cases designed for successful completion.

Agree who owns the approval policy, who reviews failures, and what evidence would justify changing the boundary. That makes “successful” mean more than getting to an answer; it means acting within the authority the team actually granted.