About this paper
A benchmark probes whether large language models can determine when imperfect simulator limitations invalidate recommended system designs, finding that identifying a limitation does not ensure correct design choices and that review-based self-correction yields mixed results.
LLM agents increasingly drive systems research and rely on simulators to evaluate candidate designs at scale. Simulators are imperfect by design: their modeling limitations can distort predictions yet leave the recommended design correct, or change which design should be selected. We introduce Siris to test whether LLMs can tell these cases apart, judging from their systems understanding when a limitation invalidates the simulator's recommendation. By perturbing a simulator, Siris constructs workload pairs with the same candidates, limitation, and simulator recommendation. The perturbation changes predictions on both workloads, yet a reference simulator supports the recommendation on only…
LLM agentslarge language modelsfrontier LLMsLLM serving design problemsself-correctionreview turn
- Upvotes
- 6
- Authors
- 6
- Published
- Nov 6, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
SIRIS: Can LLMs Tell When Simulation Errors Matter in AI-Driven Systems Research was first seen on Hugging Face · Daily papers on Nov 10, 2026. Since then it has appeared in 13 checks on 2 different days. Its best position was #11 of 30, reached on Nov 11, 05:28 UTC. It dropped off the list after Nov 11, 07:28 UTC.