DelegationBench: Measuring When AI Agents Should Ask Before Acting

Excerpt

arXiv:2610.05532v1 Announce Type: new Abstract: AI agents that send emails, edit files, and make purchases must decide when to act on their own and when to check with the user first. This decision is usually evaluated by showing a model a proposed action, asking whether it should proceed, and scoring agreement with human labels. We introduce DelegationBench to test whether such scores can be trusted. It has 156 scenarios with four possible responses (act, ask for permission, ask for missing info