Trendoday

BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design

The key-value (KV) cache dominates memory capacity and bandwidth in long-context LLM serving. While 4-bit microscaling formats (NVFP4, MXFP4) shrink the cache and run attention on FP4 Tensor Cores, a

  • Upvotes3
–Rank now
#30Peak rank
1Days on list
4Score · 7d
Nov 18, 2026First seen

About this paper

BitDecoding-FP4 co-designs FP4 quantization and system optimizations to enable accurate, high-throughput long-context KV decoding for LLMs.

The key-value (KV) cache dominates memory capacity and bandwidth in long-context LLM serving. While 4-bit microscaling formats (NVFP4, MXFP4) shrink the cache and run attention on FP4 Tensor Cores, a fundamental gap remains between algorithmic quantization and system-aware acceleration: existing FP4 KV systems use vanilla quantization that causes severe accuracy drops, whereas accuracy-oriented 4-bit algorithms rely on vector codebooks whose lookup overhead degrades decoding throughput. We present BitDecoding-FP4, an algorithm--system co-design for accurate and efficient native FP4 KV decoding. We mitigate three error sources: Dual-Residual Query Quantization splits queries into two FP4…

KV cachelong-context LLM serving4-bit microscaling formatsNVFP4MXFP4FP4 Tensor CoresFP4 KV systemsvanilla quantizationvector codebooksdecoding throughput
Upvotes
3
Authors
6
Published
Nov 6, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design was first seen on Hugging Face · Daily papers on Nov 18, 2026. Since then it has appeared in 1 check on 1 different day. Its best position was #30 of 30, reached on Nov 18, 01:28 UTC. It dropped off the list after Nov 18, 01:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 00:5801:1301:2801:4301:58 Nov 18, 01:28 UTC · #30

Last 14 days

Best rank per day (UTC).

Nov 9today

Questions

Is BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 18, 01:28 UTC, after 1 day of trending.

What is the highest rank BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design reached?

#30 on Hugging Face · Daily papers, on Nov 18, 01:28 UTC. Trendoday checks this list every 120 minutes.

How long has BitDecoding-FP4: Native FP4 KV Cache Decoding via Algorithm-System Co-Design been trending?

It was first seen on Nov 18, 2026 and has been on the list on 1 day (1 check) since.