Trendoday

Weight-Space Programming Doesn't Need Transformers

Standard neural architectures such as the Transformer were chosen because they train well with gradient descent and run fast on GPUs. In weight-space programming, a large language model (LLM) writes t

  • Upvotes11
–Rank now
#3Peak rank
2Days on list
343Score · 7d
Nov 18, 2026First seen
Rank · 7 days

About this paper

When large language models write neural network weights without gradient-based training, they tend to design sparse table or associative-memory models that outperform same-sized Transformers with far fewer parameters and faster GPU inference, though pretrained LLMs remain more accurate.

Standard neural architectures such as the Transformer were chosen because they train well with gradient descent and run fast on GPUs. In weight-space programming, a large language model (LLM) writes target weights by hand without gradient-based training, so those reasons no longer apply. Should we keep the Transformer? Given minimal guidance, ten labeled examples, and a sandbox, mainly on multiple-choice questions, we task three frontier LLMs with either generating the weights of a fixed architecture or designing the architecture themselves. When free to design, all three compile plain-text facts into a sparse table queried by a fixed procedure, reaching up to 54.4% test accuracy. An…

Transformerneural architecturesgradient descentlarge language modelweight-space programmingtarget weightsgradient-based trainingsparse tableassociative-memory-based modelpretrained weights
Upvotes
11
Authors
2
Published
Nov 17, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

Weight-Space Programming Doesn't Need Transformers was first seen on Hugging Face · Daily papers on Nov 18, 2026. Since then it has appeared in 13 checks on 2 different days. Its best position was #3 of 30, reached on Nov 18, 15:28 UTC. It dropped off the list after Nov 19, 03:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 03:2809:2815:2821:2803:28 Nov 18, 03:28 UTC · #15Nov 18, 05:28 UTC · #7Nov 18, 07:28 UTC · #6Nov 18, 09:28 UTC · #5Nov 18, 11:28 UTC · #5Nov 18, 13:28 UTC · #4Nov 18, 15:28 UTC · #3Nov 18, 17:28 UTC · #3Nov 18, 19:28 UTC · #3Nov 18, 21:28 UTC · #3Nov 18, 23:28 UTC · #3Nov 19, 01:28 UTC · #3Nov 19, 03:28 UTC · #3

Last 14 days

Best rank per day (UTC).

Nov 9today

Metric history

Platform metric as reported by Hugging Face.

0173451 03:2809:2815:2821:2803:28 Nov 18, 03:28 UTC · 11Nov 18, 05:28 UTC · 34Nov 18, 07:28 UTC · 40Nov 18, 09:28 UTC · 40Nov 18, 11:28 UTC · 40Nov 18, 13:28 UTC · 39Nov 18, 15:28 UTC · 41Nov 18, 17:28 UTC · 42Nov 18, 19:28 UTC · 47Nov 18, 21:28 UTC · 47Nov 18, 23:28 UTC · 47Nov 19, 01:28 UTC · 48Nov 19, 03:28 UTC · 51

Questions

Is Weight-Space Programming Doesn't Need Transformers trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 19, 03:28 UTC, after 2 days of trending.

What is the highest rank Weight-Space Programming Doesn't Need Transformers reached?

#3 on Hugging Face · Daily papers, on Nov 18, 15:28 UTC. Trendoday checks this list every 120 minutes.

How long has Weight-Space Programming Doesn't Need Transformers been trending?

It was first seen on Nov 18, 2026 and has been on the list on 2 days (13 checks) since.