Trendoday

DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction

Scaling depth is a key route to stronger large language models, but Pre-LN architectures often suffer from depth-wise information dilution, limiting the effective use of deep layers. While cross-layer

  • Upvotes10
–Rank now
#12Peak rank
2Days on list
229Score · 7d
Nov 11, 2026First seen
Rank · 7 days

About this paper

DepthGraft improves deep language model training by hierarchically sharing key-value representations across layers, reducing memory use while boosting accuracy.

Scaling depth is a key route to stronger large language models, but Pre-LN architectures often suffer from depth-wise information dilution, limiting the effective use of deep layers. While cross-layer connectivity has proven effective at mitigating this problem, existing designs usually introduce additional computation or memory access. We revisit cross-layer KV sharing as an efficient form of such connectivity. Beyond cache compression, each KV reuse edge creates a direct gradient path from deep layers to source KV projections, turning KV sharing into an implicit structural regularizer for KV representation geometry. We provide a systematic analysis of this mechanism, showing that proper…

Pre-LNcross-layer KV sharinggradient pathKV representation geometryDepthGrafthierarchical KV-sharingGQAMLAMoE
Upvotes
10
Authors
7
Published
Nov 2, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction was first seen on Hugging Face · Daily papers on Nov 11, 2026. Since then it has appeared in 15 checks on 2 different days. Its best position was #12 of 30, reached on Nov 11, 13:28 UTC. It dropped off the list after Nov 12, 09:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 05:2812:2819:2802:2809:28 Nov 11, 05:28 UTC · #14Nov 11, 07:28 UTC · #13Nov 11, 09:28 UTC · #15Nov 11, 11:28 UTC · #14Nov 11, 13:28 UTC · #12Nov 11, 15:28 UTC · #14Nov 11, 17:28 UTC · #15Nov 11, 19:28 UTC · #16Nov 11, 21:28 UTC · #16Nov 11, 23:28 UTC · #16Nov 12, 01:28 UTC · #16Nov 12, 03:28 UTC · #14Nov 12, 05:28 UTC · #15Nov 12, 07:28 UTC · #16Nov 12, 09:28 UTC · #15

Last 14 days

Best rank per day (UTC).

Nov 3today

Metric history

Platform metric as reported by Hugging Face.

05914 05:2812:2819:2802:2809:28 Nov 11, 05:28 UTC · 8Nov 11, 07:28 UTC · 12Nov 11, 09:28 UTC · 12Nov 11, 11:28 UTC · 12Nov 11, 13:28 UTC · 12Nov 11, 15:28 UTC · 12Nov 11, 17:28 UTC · 12Nov 11, 19:28 UTC · 12Nov 11, 21:28 UTC · 12Nov 11, 23:28 UTC · 12Nov 12, 01:28 UTC · 12Nov 12, 03:28 UTC · 14Nov 12, 05:28 UTC · 14Nov 12, 07:28 UTC · 14Nov 12, 09:28 UTC · 14

Questions

Is DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 12, 09:28 UTC, after 2 days of trending.

What is the highest rank DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction reached?

#12 on Hugging Face · Daily papers, on Nov 11, 13:28 UTC. Trendoday checks this list every 120 minutes.

How long has DepthGraft: Structural Regularization through Hierarchical Cross-Layer KV Reconstruction been trending?

It was first seen on Nov 11, 2026 and has been on the list on 2 days (15 checks) since.