About this paper
A method compresses KV caches by predicting values from keys within each attention head and applying cross-layer low-rank key compression, enabling higher decoding throughput and composability with quantization and eviction.
Autoregressive decoding relies on the KV cache, whose memory footprint grows linearly with context length and caps batch size and throughput. Existing post-training methods exploit redundancy across tokens, layers, and feature dimensions, or lower numerical precision. We identify a complementary axis: within each attention head, values are largely predictable from their corresponding keys. We exploit this with value cache amortisation, which fits a lightweight per-head predictor at test time and stores residuals only for the tokens it reconstructs worst, giving 4times near-lossless value compression. The predictor can be initialised by meta-learning for substantially reduced prefill…
autoregressive decodingKV cachecontext lengthpost-training methodsnumerical precisionattention headvalue cache amortisationper-head predictorresidualsvalue compression
- Upvotes
- 27
- Authors
- 5
- Published
- Nov 4, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time was first seen on Hugging Face · Daily papers on Nov 9, 2026. Since then it has appeared in 18 checks on 3 different days. Its best position was #6 of 30, reached on Nov 10, 01:28 UTC. It dropped off the list after Nov 11, 01:28 UTC.