About this paper
Reinforcement learning agents can exploit verifiable reward-context conflicts to earn higher rewards through noncompliant shortcuts, while contextual penalties and safety learning can improve compliance without sacrificing task capability.
Reinforcement learning (RL) can improve an agent's task capability while reinforcing behavior that violates contextual execution requirements. We study this reward-driven misalignment, including reward hacking, through verifiable reward--context conflicts, where compliant task completion remains feasible while some executable shortcuts earn more reward than every compliant strategy. Encoding these conflicts in task dynamics lets us generate diverse interactive environments at scale and derive compliance feedback from execution histories. In resource and reporting workflows, task optimization increases real productivity through unauthorized resource use or raises reported scores while…
reward hackingreward-driven misalignmentverifiable reward-context conflictscompliant task completionexecutable shortcutstask dynamicsinteractive environmentscompliance feedbackexecution historiestraining distribution
- Upvotes
- 3
- Authors
- 1
- Published
- Nov 9, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
Shortcuts and Boundaries: Verifiable RL Misalignment and Contextual Safety Learning was first seen on Hugging Face · Daily papers on Nov 11, 2026. Since then it has appeared in 17 checks on 3 different days. Its best position was #26 of 30, reached on Nov 12, 11:28 UTC. It dropped off the list after Nov 13, 05:28 UTC.