Auriga, a task-agnostic agent harness, closes most of the planning-execution gap in zero-shot robot manipulation by combining explicit task beliefs, calibrated 3D measurements, dual-arm control, and execution monitoring, achieving state-of-the-art results on RoboDojo and RoboCasa365.
Vision-language models can interpret instructions and plan manipulation tasks, yet struggle when acting directly as robot policies. On RoboDojo, a large planning-execution gap separates high-level task decisions from physical execution: removing arm execution raises GPT-6 Astra's success rate from 22.5% under the reference control interface to 76.2%. We present Auriga, a task-agnostic agent harness that closes 68.3% of this gap in zero-shot manipulation without target-task trials or resets. In Auriga, the model makes all task decisions over an explicit task belief and acts through either direct tool calls or synthesized multi-step Python programs, while the harness supplies calibrated 3D…
vision-language modelsrobot policieszero-shot manipulationtask-agnostic agent harnessplanning-execution gaptask belieftool callssynthesized multi-step Python programscalibrated 3D measurementsfinger-referenced dual-arm control
- Upvotes
- 4
- Authors
- 21
- Published
- Oct 31, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.