Trendoday

TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates

Hyper-Connections (HC) expand the residual stream of a Transformer into N parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale. However, mHC benefits little from more than four

  • Upvotes8
–Rank now
#26Peak rank
1Days on list
23Score · 7d
Nov 18, 2026First seen
Rank · 7 days

About this paper

TPHC combines tensor-parallel shards with hyper-connection residual streams to enable more independent stream updates, improving language model loss and zero-shot reasoning with modest training overhead.

Hyper-Connections (HC) expand the residual stream of a Transformer into N parallel streams, and manifold-constrained HC (mHC) stabilizes them at scale. However, mHC benefits little from more than four streams, partly because each sublayer writes a single output to all streams, limiting how differently they can be updated. In this work, we propose TPHC, which integrates tensor-parallel (TP) shards with HC streams to promote greater independence across streams. Specifically, we route a separate mixture of the residual streams to each TP shard and keep the individual shard outputs separate during write-back, so each stream can receive a different mixture of those outputs. This design makes…

Hyper-Connectionsresidual streamTransformermanifold-constrained HCtensor-parallel shardsHC streamsmixture of residual streamswrite-backattention sublayersFFN sublayers
Upvotes
8
Authors
7
Published
Nov 17, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates was first seen on Hugging Face · Daily papers on Nov 18, 2026. Since then it has appeared in 2 checks on 1 different day. Its best position was #26 of 30, reached on Nov 18, 09:28 UTC. It dropped off the list after Nov 18, 11:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 09:2809:5810:2810:5811:28 Nov 18, 09:28 UTC · #26Nov 18, 11:28 UTC · #27

Last 14 days

Best rank per day (UTC).

Nov 9today

Metric history

Platform metric as reported by Hugging Face.

0358 09:2809:5810:2810:5811:28 Nov 18, 09:28 UTC · 8Nov 18, 11:28 UTC · 8

Questions

Is TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 18, 11:28 UTC, after 1 day of trending.

What is the highest rank TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates reached?

#26 on Hugging Face · Daily papers, on Nov 18, 09:28 UTC. Trendoday checks this list every 120 minutes.

How long has TPHC: Improving Hyper-Connections with Tensor-Parallel Grouped Updates been trending?

It was first seen on Nov 18, 2026 and has been on the list on 1 day (2 checks) since.