Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout execution or evaluate task outcomes
REALCLI is a benchmark that jointly evaluates CLI agent task execution and human-agent collaboration quality, revealing major gaps in steerability, trust calibration, and safety management.
Command-line interface (CLI) agents increasingly execute longer-horizon tasks, yet existing benchmarks either assume that user instructions remain fixed throughout execution or evaluate task outcomes without capturing runtime collaboration quality. In this paper, we present REALCLI, a paired benchmark that unifies the evaluation of task execution and human-agent collaboration quality on realistic CLI tasks. To comprehensively measure collaboration, REALCLI builds a five-dimensional framework, operationalized into 22 sub-dimensions and 51 interaction scenario templates. REALCLI provides two modes for paired comparison: REALCLI-EXEC supplies complete specifications upfront, whereas…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
RealCLI: Toward a Unified Evaluation of Human-Agent Collaboration for CLI Agents was first seen on Hugging Face · Daily papers on Oct 29, 2026. Since then it has appeared in 43 checks on 5 different days. Its best position was #11 of 30, reached on Oct 30, 05:28 UTC. It dropped off the list after Nov 2, 01:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 2, 01:28 UTC, after 5 days of trending.
#11 on Hugging Face · Daily papers, on Oct 30, 05:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Oct 29, 2026 and has been on the list on 5 days (43 checks) since.