Library of Layers dynamically orders transformer blocks during inference to adapt computation to tasks, enabling parameter reuse, sparsity, and interpretable routing across reasoning, multilingual, and safety settings.
Computer programmers chain together the same reusable computational primitives, for example, conditional statements or loops, but in different orders to implement different algorithms. In contrast, existing language model architectures always execute their layers in the same order. We introduce Library of Layers (LoL), an architecture that dynamically orders its layers during inference to solve the problem at hand. LoL has a single hyperparameter, the size of its global library of transformer blocks, that allows it to transition seamlessly between parameter reuse and sparsity, generalizing the parameter-efficiency and FLOPs-efficiency tradeoffs of dense, depth-recurrent, and mixture of…
Library of Layersdynamic layer orderingtransformer blocksparameter reusesparsitymixture of expertsdepth-recurrentroutingrouter outputs
- Upvotes
- 6
- Authors
- 6
- Published
- Nov 2, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.