Bf16 might be freezing your weight updates
2 comments
- Points8
- Comments2
–Rank now
#29Peak rank
1Days on list
3Score · 7d
Nov 5, 2026First seen
Rank · 7 days
On Hacker News
- Points
- 8
- Comments
- 2
- Submitted by
- martincsweiss
- Posted
- Nov 2, 2026
- Site
- lacuna.tiptreesystems.com
Story details from Hacker News ↗ · linked article on lacuna.tiptreesystems.com.
Trending history
Bf16 might be freezing your weight updates was first seen on Hacker News · Front page on Nov 5, 2026. Since then it has appeared in 3 checks on 1 different day. Its best position was #29 of 30, reached on Nov 5, 01:07 UTC. It dropped off the list after Nov 5, 01:07 UTC.
Rank history
Position on Hacker News · Front page at every check (lower is better).
Last 14 days
Best rank per day (UTC).
29
Oct 26today
Metric history
Platform metric as reported by Hacker News.
From the discussion
A few top-level comments from Hacker News, shortened. Read the full thread on Hacker News.
There's a reason we keep around an fp32 copy of the weights? This issue was recognized a decade ago when we first started training with fp16.
If you fine-tune a model with weights stored in bf16 and let torch.optim.AdamW write to them directly, you get interesting behaviour. bf16 can only represent changes to weights if the change is at least ~1/512 of the weight itself, so at small learning rates only the smallest weights can update. As a result, you have…
Featured in
Archived daily, weekly and monthly lists that include this entry.
Hacker News 1 day, best #29
Questions
Is Bf16 might be freezing your weight updates trending on Hacker News right now?
Not at the moment. It was last on the list on Nov 5, 01:07 UTC, after 1 day of trending.
What is the highest rank Bf16 might be freezing your weight updates reached?
#29 on Hacker News · Front page, on Nov 5, 01:07 UTC. Trendoday checks this list every 15 minutes.
How long has Bf16 might be freezing your weight updates been trending?
It was first seen on Nov 5, 2026 and has been on the list on 1 day (3 checks) since.
How much discussion did it get on Hacker News?
8 points and 2 comments; submitted by martincsweiss on Nov 2, 2026.