Trendoday

onóna: Twin Embeddings with Tied Unembeddings

Weight tying reduces the parameter cost of language models, but the usual tied output head is limited to a single softmax. We introduce onóna, a two-table parameterization that increases output expres

  • Upvotes3
–Rank now
#30Peak rank
1Days on list
4Score · 7d
Nov 11, 2026First seen

About this paper

A two-table weight-sharing method for Transformer language models improves output expressiveness and reduces validation loss and perplexity across several architectures and scales at the cost of an extra embedding table and vocabulary projection.

Weight tying reduces the parameter cost of language models, but the usual tied output head is limited to a single softmax. We introduce onóna, a two-table parameterization that increases output expressiveness while preserving input-output weight sharing. The Transformer takes the average of two embedding tables as input, and the output head combines the softmax distributions defined by those same tables. A gate conditioned only on the current input token determines the mixture weight. Its values can be precomputed for inference, and the Transformer body remains unchanged. We evaluate onóna across Dense, Gated DeltaNet hybrid, mixture-of-experts, and Attention Residual architectures at…

weight tyinglanguage modelstied output headsoftmaxtwo-table parameterizationinput-output weight sharingTransformerembedding tablesoutput headsoftmax distributions
Upvotes
3
Authors
7
Published
Nov 7, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

onóna: Twin Embeddings with Tied Unembeddings was first seen on Hugging Face · Daily papers on Nov 11, 2026. Since then it has appeared in 1 check on 1 different day. Its best position was #30 of 30, reached on Nov 11, 03:28 UTC. It dropped off the list after Nov 11, 03:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 02:5803:1303:2803:4303:58 Nov 11, 03:28 UTC · #30

Last 14 days

Best rank per day (UTC).

Nov 3today

Questions

Is onóna: Twin Embeddings with Tied Unembeddings trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 11, 03:28 UTC, after 1 day of trending.

What is the highest rank onóna: Twin Embeddings with Tied Unembeddings reached?

#30 on Hugging Face · Daily papers, on Nov 11, 03:28 UTC. Trendoday checks this list every 120 minutes.

How long has onóna: Twin Embeddings with Tied Unembeddings been trending?

It was first seen on Nov 11, 2026 and has been on the list on 1 day (1 check) since.