Proving and Improving the Platonic Representation Hypothesis with JEPAs

For those reading the paper: the positives of a good-view JEPA are described as “two observations of one scene,” and yet the paper also states that “A good-view JEPA's positives are (g_m(z, ν), g_m(z′, ν′)) with (z, z′) ∼ π and independent nuisances.” My question is: why doesn't “two observations of one scene” mean…

I'm not sure how I feel about the Platonic Representation Hypothesis. In some ways it seems like it should be profound, but part of me thinks it might just be saying 'There is an optimal way to perform dimensionality reduction and as things get bigger it is easier to appoximate'

> 33M-parameter text JEPA reaches .125 on the PRH paper’s benchmark, against .034 on Flickr captions, level with Qwen2.5-3B and -7B. They need fewer samples. Im bullish on jepa models. Success of autoregressive next token prediction feels like a stroke of luck.