Trendoday

Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds

Visual interaction models that support users during ongoing physical tasks require evaluation in closed loop, where assistant responses can change subsequent user behavior and task outcomes. Existing

  • Upvotes13
–Rank now
#14Peak rank
5Days on list
303Score · 7d
Nov 12, 2026First seen
Rank · 7 days

About this paper

Interactive Visual Gym enables closed-loop evaluation and training of visual interaction models by jointly simulating task environments and human users, exposing limitations in state-of-the-art VLMs and improving performance through closed-loop training.

Visual interaction models that support users during ongoing physical tasks require evaluation in closed loop, where assistant responses can change subsequent user behavior and task outcomes. Existing benchmarks largely rely on offline replay and therefore cannot capture these effects. We introduce Interactive Visual Gym, a framework for closed-loop evaluation and training of visual interaction models through joint simulation of executable task environments and human users. We contribute three simulators that recreate everyday tasks and human-aligned action spaces across cooking, household manipulation, and mobile-interface domains, together with a persona-conditioned user simulation…

visual interaction modelsclosed-loop evaluationclosed-loop trainingjoint simulationexecutable task environmentshuman-aligned action spacespersona-conditioned user simulationout-of-plan behaviorsIVG-Benchvision-language models (VLMs)
Upvotes
13
Authors
14
Published
Nov 5, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds was first seen on Hugging Face · Daily papers on Nov 12, 2026. Since then it has appeared in 41 checks on 5 different days. Its best position was #14 of 30, reached on Nov 13, 07:28 UTC. It dropped off the list after Nov 16, 01:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 Nov 12Nov 13Nov 14Nov 15Nov 16 Nov 12, 17:28 UTC · #29Nov 12, 19:28 UTC · #30Nov 12, 21:28 UTC · #23Nov 12, 23:28 UTC · #16Nov 13, 01:28 UTC · #15Nov 13, 03:28 UTC · #15Nov 13, 05:28 UTC · #15Nov 13, 07:28 UTC · #14Nov 13, 09:28 UTC · #18Nov 13, 11:28 UTC · #18Nov 13, 13:28 UTC · #19Nov 13, 15:28 UTC · #19Nov 13, 17:28 UTC · #16Nov 13, 19:28 UTC · #16Nov 13, 21:28 UTC · #15Nov 13, 23:28 UTC · #15Nov 14, 01:28 UTC · #15Nov 14, 03:28 UTC · #15Nov 14, 05:28 UTC · #16Nov 14, 07:28 UTC · #17Nov 14, 09:28 UTC · #17Nov 14, 11:28 UTC · #17Nov 14, 13:28 UTC · #17Nov 14, 15:28 UTC · #17Nov 14, 17:28 UTC · #17Nov 14, 19:28 UTC · #17Nov 14, 21:28 UTC · #17Nov 14, 23:28 UTC · #18Nov 15, 01:28 UTC · #18Nov 15, 03:28 UTC · #18Nov 15, 05:28 UTC · #18Nov 15, 07:28 UTC · #18Nov 15, 09:28 UTC · #18Nov 15, 11:28 UTC · #19Nov 15, 13:28 UTC · #19Nov 15, 15:28 UTC · #19Nov 15, 17:28 UTC · #19Nov 15, 19:28 UTC · #19Nov 15, 21:28 UTC · #19Nov 15, 23:28 UTC · #20Nov 16, 01:28 UTC · #20

Last 14 days

Best rank per day (UTC).

Nov 3today

Metric history

Platform metric as reported by Hugging Face.

04913 Nov 12Nov 13Nov 14Nov 15Nov 16 Nov 12, 17:28 UTC · 4Nov 12, 19:28 UTC · 4Nov 12, 21:28 UTC · 6Nov 12, 23:28 UTC · 13Nov 13, 01:28 UTC · 13Nov 13, 03:28 UTC · 13Nov 13, 05:28 UTC · 13Nov 13, 07:28 UTC · 13Nov 13, 09:28 UTC · 13Nov 13, 11:28 UTC · 13Nov 13, 13:28 UTC · 13Nov 13, 15:28 UTC · 13Nov 13, 17:28 UTC · 13Nov 13, 19:28 UTC · 13Nov 13, 21:28 UTC · 13Nov 13, 23:28 UTC · 13Nov 14, 01:28 UTC · 13Nov 14, 03:28 UTC · 13Nov 14, 05:28 UTC · 13Nov 14, 07:28 UTC · 13Nov 14, 09:28 UTC · 13Nov 14, 11:28 UTC · 13Nov 14, 13:28 UTC · 13Nov 14, 15:28 UTC · 13Nov 14, 17:28 UTC · 13Nov 14, 19:28 UTC · 13Nov 14, 21:28 UTC · 13Nov 14, 23:28 UTC · 13Nov 15, 01:28 UTC · 13Nov 15, 03:28 UTC · 13Nov 15, 05:28 UTC · 13Nov 15, 07:28 UTC · 13Nov 15, 09:28 UTC · 13Nov 15, 11:28 UTC · 13Nov 15, 13:28 UTC · 13Nov 15, 15:28 UTC · 13Nov 15, 17:28 UTC · 13Nov 15, 19:28 UTC · 13Nov 15, 21:28 UTC · 13Nov 15, 23:28 UTC · 13Nov 16, 01:28 UTC · 13

Questions

Is Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 16, 01:28 UTC, after 5 days of trending.

What is the highest rank Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds reached?

#14 on Hugging Face · Daily papers, on Nov 13, 07:28 UTC. Trendoday checks this list every 120 minutes.

How long has Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds been trending?

It was first seen on Nov 12, 2026 and has been on the list on 5 days (41 checks) since.