JVT, a simple large-patch DiT-style Transformer, enables pixel-space video diffusion without pretrained autoencoders or U-Net skip connections, achieving competitive quality and supporting joint camera-conditioned RGB-disparity generation.
Direct diffusion modeling of high-dimensional raw video data is challenging, and existing pixel-space methods often rely on specialized architectures. We introduce JVT (Just Video Transformer), a simple large-patch DiT-style Transformer for pixel-space video diffusion, without a pretrained autoencoder, auxiliary detailers, cascaded stages, or U-Net-style skip connections. We systematically examine video patch embeddings, register tokens, and adaptive layer normalization to identify design choices that balance generation quality and computational efficiency. The resulting JVT design achieves generation quality competitive with latent diffusion baselines in both class-conditioned and…
pixel-space video diffusionDiT-style Transformerlarge-patch Transformerpretrained autoencoderauxiliary detailerscascaded stagesU-Net-style skip connectionsvideo patch embeddingsregister tokensadaptive layer normalization
- Upvotes
- 12
- Authors
- 3
- Published
- Nov 13, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.