About this paper
We present K2-Horizon, a fleet of six fully open language models, from a 0.9B dense model to a 375B-A23B MoE, built for general-purpose reasoning and agentic work across math and science, software engineering, deep research, office automation, and beyond. The models lead or are competitive with strong open models of their size on agentic and coding benchmarks. On the modeling side, to strengthen reasoning, we introduce Mixture-of-Value Attention (MoVA), applying MoE to attention value projections, and softplus attention gating, a more flexible, theoretically motivated gate. Our 36B-A4B model uses both and nearly matches a 32B dense model trained on the same data, with one-seventh the…
- Upvotes
- 27
- Authors
- 95
- Published
- Nov 16, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
K2-Horizon: A Frontier Fleet of Open Source Language Models was first seen on Hugging Face · Daily papers on Nov 17, 2026. Since then it has appeared in 12 checks on 2 different days. Its best position was #2 of 30, reached on Nov 17, 11:28 UTC. It dropped off the list after Nov 18, 05:28 UTC.