About this paper
Large language models struggle with cumulative state tracking across multiple updates despite strong single-step arithmetic, and their measured accuracy depends heavily on output protocols.
Human working memory lets people maintain and update information. Large language models (LLMs) face a related demand when tracking updates within a single prompt without writing out intermediate states, yet existing tests differ in their constraints on written steps. We introduce a working-memory-inspired probe of cumulative state tracking with program-computed answers, 3 to 100 updates, and final-state, intermediate-state, or multi-target queries. It compares protocols on identical items. Across 18 LLMs, accuracy is 84.9% on single-step arithmetic but 31.7% on core items requiring one number after three to seven updates. From 50 updates on, no model exceeds 10% accuracy. Core accuracy…
working memorycumulative state trackinglarge language modelsmulti-target queriesBerkeley Function Calling Leaderboard
- Upvotes
- 4
- Authors
- 6
- Published
- Oct 30, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
WMIP: A Working-Memory-Inspired Programmable Probe of Cumulative State Tracking in Large Language Models was first seen on Hugging Face · Daily papers on Nov 13, 2026. Since then it has appeared in 7 checks on 1 different day. Its best position was #27 of 30, reached on Nov 13, 23:28 UTC. It dropped off the list after Nov 13, 23:28 UTC.