About this paper
A decision model that acts poorly can still teach reinforcement learning policies effectively by judging complete action trajectories, improving success on ALFWorld and WebShop.
Decision models answer a question with a typed output, such as a choice among options with a probability for each, so one model can choose an agent's actions, evaluate actions already taken, or score every action to teach a policy with reinforcement learning. Prior work has studied them mainly as actors, judging their usefulness by how well they act. We ask whether a decision model that acts poorly can still teach well. We compare three decision models and three general LLMs as actors and judges on ALFWorld, and use JEV as a teacher on ALFWorld and WebShop. As actors, all six models are pulled away from a subgoal that must come first by an easier one. As judges that see the whole…
decision modelstyped outputreinforcement learningpolicyactorsjudgestrajectoriessubgoalgeneral LLMsALFWorld
- Upvotes
- 12
- Authors
- 10
- Published
- Nov 12, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents was first seen on Hugging Face · Daily papers on Nov 13, 2026. Since then it has appeared in 36 checks on 4 different days. Its best position was #7 of 30, reached on Nov 16, 05:28 UTC. It is #7 right now, climbing from #9 to #7 over the last 6 checks.