HALO introduces a human-annotated benchmark and dataset to evaluate proactive agents, revealing that over-proactivity is the dominant failure mode.
Proactive agents must decide whether, when, and how far to assist without an explicit request. Existing evaluations often rely on automated scores or binary verdicts, offering limited human diagnosis of evidence use and intervention quality. To address these gaps, we introduce HALO, Human-Grounded GrAnular EvaLuation of PrOactive Agents. HALO-Arena covers 4 proactive need types, Dependency, Blockage, Risk, and Opportunity, across 10 everyday domains and 50 personal environments. Scenarios require agents to retrieve evidence from structured databases or unstructured documents before judging whether assistance is warranted and choosing its scope. On HALO-Arena, we construct HALO-8K, a…
proactive agentsHALO-ArenaHALO-8Kevidence retrievalover-proactivityaction levelacceptability verdictsreasoning tokensretrieval rounds
- Upvotes
- 6
- Authors
- 8
- Published
- Oct 29, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.