Trendoday

Multi-Branch Transformer

Transformer language models are commonly scaled by making their blocks wider or stacking them deeper. In this work, we explore a complementary path: branching, the parallel composition of full Transfo

  • Upvotes16
–Rank now
#4Peak rank
2Days on list
328Score · 7d
Nov 18, 2026First seen
Rank · 7 days

About this paper

Branching Transformer blocks into parallel parameterized branches, combined with shared KV caching, improves language model quality and inference efficiency compared with standard sequential Transformers.

Transformer language models are commonly scaled by making their blocks wider or stacking them deeper. In this work, we explore a complementary path: branching, the parallel composition of full Transformer blocks. We introduce the Multi-Branch Transformer (MBT), which processes the input through shared initial layers, executes independently parameterized branches of sequential blocks in parallel, and fuses their outputs before shared final layers. At comparable training compute, we study three MBT variants: Wide-Shallow preserves block width, Narrow-Deep preserves sequential depth, and Balanced interpolates between them. In controlled pretraining experiments at four mixture-of-experts…

Multi-Branch Transformer (MBT)branchingparallel composition of Transformer blocksshared initial layersindependently parameterized branchessequential blocksoutput fusionshared final layersWide-ShallowNarrow-Deep
Upvotes
16
Authors
9
Published
Nov 15, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

Multi-Branch Transformer was first seen on Hugging Face · Daily papers on Nov 18, 2026. Since then it has appeared in 13 checks on 2 different days. Its best position was #4 of 30, reached on Nov 18, 15:28 UTC. It dropped off the list after Nov 19, 03:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 03:2809:2815:2821:2803:28 Nov 18, 03:28 UTC · #11Nov 18, 05:28 UTC · #9Nov 18, 07:28 UTC · #8Nov 18, 09:28 UTC · #7Nov 18, 11:28 UTC · #9Nov 18, 13:28 UTC · #6Nov 18, 15:28 UTC · #4Nov 18, 17:28 UTC · #4Nov 18, 19:28 UTC · #4Nov 18, 21:28 UTC · #4Nov 18, 23:28 UTC · #4Nov 19, 01:28 UTC · #4Nov 19, 03:28 UTC · #4

Last 14 days

Best rank per day (UTC).

Nov 9today

Metric history

Platform metric as reported by Hugging Face.

0153045 03:2809:2815:2821:2803:28 Nov 18, 03:28 UTC · 16Nov 18, 05:28 UTC · 23Nov 18, 07:28 UTC · 28Nov 18, 09:28 UTC · 31Nov 18, 11:28 UTC · 31Nov 18, 13:28 UTC · 33Nov 18, 15:28 UTC · 39Nov 18, 17:28 UTC · 41Nov 18, 19:28 UTC · 41Nov 18, 21:28 UTC · 43Nov 18, 23:28 UTC · 43Nov 19, 01:28 UTC · 44Nov 19, 03:28 UTC · 45

Questions

Is Multi-Branch Transformer trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 19, 03:28 UTC, after 2 days of trending.

What is the highest rank Multi-Branch Transformer reached?

#4 on Hugging Face · Daily papers, on Nov 18, 15:28 UTC. Trendoday checks this list every 120 minutes.

How long has Multi-Branch Transformer been trending?

It was first seen on Nov 18, 2026 and has been on the list on 2 days (13 checks) since.