Trendoday

On the Design of Hyper-Connections: Simplification, Stability, and Scalability

Hyper-Connections (HC) architectures are increasingly adopted in frontier language models, yet their components remain insufficiently understood and introduce non-negligible training overhead. In this

  • Upvotes29
#10Rank now –
#8Peak rank
4Days on list
455Score · 7d
Nov 13, 2026First seen
Rank · 7 days

About this paper

A systematic study of Hyper-Connections architectures identifies the simpler efficient design SimpleHC, which removes residual mixing and uses shared memory access to match mHC performance while adding minimal training overhead and improving stability.

Hyper-Connections (HC) architectures are increasingly adopted in frontier language models, yet their components remain insufficiently understood and introduce non-negligible training overhead. In this work, we systematically study the HC architecture to understand its components and identify simpler, more efficient designs. We find that learned read/write gating without mixing and unconstrained mixing with fixed gates form two effective interaction groups: either alone achieves performance comparable to mHC. We also identify repeated access to the expanded residual state as a major source of training overhead, even without mixing. These findings lead to SimpleHC, which simplifies the…

Hyper-Connections architecturesfrontier language modelslearned read/write gatingunconstrained mixingfixed gatesmHCexpanded residual stateSimpleHCresidual mixingshared memory access
Upvotes
29
Authors
11
Published
Nov 4, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

On the Design of Hyper-Connections: Simplification, Stability, and Scalability was first seen on Hugging Face · Daily papers on Nov 13, 2026. Since then it has appeared in 37 checks on 4 different days. Its best position was #8 of 30, reached on Nov 13, 07:28 UTC. It is #10 right now, climbing from #11 to #10 over the last 6 checks.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 Nov 13Nov 13Nov 14Nov 15Nov 16 Nov 13, 05:28 UTC · #14Nov 13, 07:28 UTC · #8Nov 13, 09:28 UTC · #11Nov 13, 11:28 UTC · #10Nov 13, 13:28 UTC · #11Nov 13, 15:28 UTC · #11Nov 13, 17:28 UTC · #10Nov 13, 19:28 UTC · #10Nov 13, 21:28 UTC · #10Nov 13, 23:28 UTC · #10Nov 14, 01:28 UTC · #10Nov 14, 03:28 UTC · #11Nov 14, 05:28 UTC · #11Nov 14, 07:28 UTC · #11Nov 14, 09:28 UTC · #11Nov 14, 11:28 UTC · #11Nov 14, 13:28 UTC · #10Nov 14, 15:28 UTC · #10Nov 14, 17:28 UTC · #10Nov 14, 19:28 UTC · #10Nov 14, 21:28 UTC · #10Nov 14, 23:28 UTC · #10Nov 15, 01:28 UTC · #10Nov 15, 03:28 UTC · #10Nov 15, 05:28 UTC · #10Nov 15, 07:28 UTC · #10Nov 15, 09:28 UTC · #10Nov 15, 11:28 UTC · #10Nov 15, 13:28 UTC · #10Nov 15, 15:28 UTC · #11Nov 15, 17:28 UTC · #11Nov 15, 19:28 UTC · #11Nov 15, 21:28 UTC · #11Nov 15, 23:28 UTC · #11Nov 16, 01:28 UTC · #11Nov 16, 03:28 UTC · #10Nov 16, 05:28 UTC · #10

Last 14 days

Best rank per day (UTC).

Nov 3today

Metric history

▲ as reported by Hugging Face.

0101929 Nov 13Nov 13Nov 14Nov 15Nov 16 Nov 13, 05:28 UTC · 13Nov 13, 07:28 UTC · 22Nov 13, 09:28 UTC · 22Nov 13, 11:28 UTC · 24Nov 13, 13:28 UTC · 25Nov 13, 15:28 UTC · 26Nov 13, 17:28 UTC · 26Nov 13, 19:28 UTC · 26Nov 13, 21:28 UTC · 26Nov 13, 23:28 UTC · 26Nov 14, 01:28 UTC · 26Nov 14, 03:28 UTC · 26Nov 14, 05:28 UTC · 26Nov 14, 07:28 UTC · 26Nov 14, 09:28 UTC · 26Nov 14, 11:28 UTC · 26Nov 14, 13:28 UTC · 27Nov 14, 15:28 UTC · 27Nov 14, 17:28 UTC · 27Nov 14, 19:28 UTC · 27Nov 14, 21:28 UTC · 27Nov 14, 23:28 UTC · 29Nov 15, 01:28 UTC · 29Nov 15, 03:28 UTC · 29Nov 15, 05:28 UTC · 29Nov 15, 07:28 UTC · 29Nov 15, 09:28 UTC · 29Nov 15, 11:28 UTC · 29Nov 15, 13:28 UTC · 29Nov 15, 15:28 UTC · 29Nov 15, 17:28 UTC · 29Nov 15, 19:28 UTC · 29Nov 15, 21:28 UTC · 29Nov 15, 23:28 UTC · 29Nov 16, 01:28 UTC · 29Nov 16, 03:28 UTC · 29Nov 16, 05:28 UTC · 29

Questions

Is On the Design of Hyper-Connections: Simplification, Stability, and Scalability trending on HF Daily Papers right now?

Yes. As of Nov 16, 05:28 UTC it is #10, climbing from #11 to #10 over the last 6 checks.

What is the highest rank On the Design of Hyper-Connections: Simplification, Stability, and Scalability reached?

#8 on Hugging Face · Daily papers, on Nov 13, 07:28 UTC. Trendoday checks this list every 120 minutes.

How long has On the Design of Hyper-Connections: Simplification, Stability, and Scalability been trending?

It was first seen on Nov 13, 2026 and has been on the list on 4 days (37 checks) since.