About this paper
A controlled two-pass audit varying only the revision context reveals that language models often fail to selectively correct errors under disagreement, with expert labels increasing reversals but not reliably improving accuracy, so revision should be assessed using paired conditional metrics per…
When language models revise a decision after another agent disagrees, neither reversal counts nor final accuracy shows whether they are fixing mistakes or abandoning correct answers. We introduce a controlled two-pass audit that holds the question, the evidence, and the model's initial answer fixed and varies only the revision context: neutral reflection, anonymous disagreement, or disagreement attributed to a domain expert. For binary decisions the accounting is exact: revision raises accuracy only if it is selective enough, reversing wrong answers more than a/(1-a) times as often as correct ones, where a is initial accuracy (nine times at 90%). Across five models and 1,989…
language modelscontrolled two-pass auditrevision contextneutral reflectionanonymous disagreementexpert disagreementbinary decisionsselective revisioninitial accuracyreversal counts
- Upvotes
- 3
- Authors
- 2
- Published
- Nov 12, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
When Expert Disagreement Hurts: Prestige and Selectivity in LLM Decision Revision was first seen on Hugging Face · Daily papers on Nov 17, 2026. Since then it has appeared in 4 checks on 2 different days. Its best position was #28 of 30, reached on Nov 18, 15:28 UTC. It dropped off the list after Nov 18, 15:28 UTC.