About this paper
A benchmark of dated U.S. financial facts shows that large language models often understate their knowledge and that common safeguards cannot reliably certify that historical backtests are free of information leakage.
Financial backtests of large language models (LLMs) are meaningful only if the model did not already know the outcomes it is asked to predict. Practitioners guard against this by trusting a vendor's stated knowledge cutoff, by asking the model what it knows, or by instructing it to answer as of a past date. To verify whether any of these safeguards can be trusted, we introduce FinDendro, a benchmark of 2,226 dated U.S. facts from 1999 to 2026, covering monthly macroeconomic series, month-end stock prices and Federal Reserve decisions. Each fact yields four prompts: a value question, Recall and Predict questions that pair each candidate value with a claim of knowing or guessing without a…
large language models (LLMs)knowledge cutofffrontier modelsself-reported knowledgemodel training windowtemporal knowledge boundariesleak-free backtestsFinDendro benchmark
- Upvotes
- 16
- Authors
- 15
- Published
- Nov 4, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
LLMs Don't Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting was first seen on Hugging Face · Daily papers on Nov 9, 2026. Since then it has appeared in 11 checks on 2 different days. Its best position was #8 of 30, reached on Nov 10, 05:28 UTC. It dropped off the list after Nov 10, 13:28 UTC.