Trendoday

Are Dense Rewards Worth Their Tokens in GRPO?

RLVR trains language models to reason by rewarding correct final answers. A widely used algorithm, group-relative policy optimization (GRPO), gives every token of a solution the same credit, so dense

  • Upvotes3
–Rank now
#30Peak rank
2Days on list
5Score · 7d
Nov 11, 2026First seen
Rank · 7 days

About this paper

At matched compute, dense process rewards such as Monte-Carlo rewards underperform standard GRPO for language-model reasoning, while selecting prompts with predicted success near 1/2 yields small consistent accuracy gains.

RLVR trains language models to reason by rewarding correct final answers. A widely used algorithm, group-relative policy optimization (GRPO), gives every token of a solution the same credit, so dense process rewards have been proposed to credit intermediate steps, for example with a Monte-Carlo reward that scores each step by sampling continuations from it. Comparing such a reward with GRPO per training update, or vetting it with a gradient test on a single batch, leaves out what it costs: in our setting, the continuations alone take 91% of all generated tokens. We introduce a matched-compute protocol that gives every training intervention the same budget, measured in generated tokens and…

RLVRgroup-relative policy optimization (GRPO)dense process rewardsMonte-Carlo rewardmatched-compute protocolper-token gradient testdense-reward variantslearning rateprompt selection by predicted success near 1/2Qwen2.5-1.5B
Upvotes
3
Authors
19
Published
Nov 10, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

Are Dense Rewards Worth Their Tokens in GRPO? was first seen on Hugging Face · Daily papers on Nov 11, 2026. Since then it has appeared in 2 checks on 2 different days. Its best position was #30 of 30, reached on Nov 11, 13:28 UTC. It dropped off the list after Nov 12, 11:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 13:2818:5800:2805:5811:28 Nov 11, 13:28 UTC · #30Nov 12, 11:28 UTC · #30

Last 14 days

Best rank per day (UTC).

Nov 3today

Metric history

Platform metric as reported by Hugging Face.

0134 13:2818:5800:2805:5811:28 Nov 11, 13:28 UTC · 3Nov 12, 11:28 UTC · 4

Questions

Is Are Dense Rewards Worth Their Tokens in GRPO? trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 12, 11:28 UTC, after 2 days of trending.

What is the highest rank Are Dense Rewards Worth Their Tokens in GRPO? reached?

#30 on Hugging Face · Daily papers, on Nov 11, 13:28 UTC. Trendoday checks this list every 120 minutes.

How long has Are Dense Rewards Worth Their Tokens in GRPO? been trending?

It was first seen on Nov 11, 2026 and has been on the list on 2 days (2 checks) since.