This unified foundation model enables the recovery of dense geometry, camera poses, and object tracking from diverse, uncalibrated observations in 4D scenes using a shared geometry and motion branch architecture.
π^4 is a unified foundation model for 4D scene perception. From diverse, uncalibrated RGB observations in the wild---a single image, an unordered collection, or a video sequence---it recovers dense geometry, camera poses, and metric scale, together with 2D/3D point tracks and moving-object masks for multi-image inputs. It couples a geometry branch with a geometry-conditioned motion branch, which predicts image-space correspondences that are lifted into 3D using reconstructed target-view geometry. Across diverse benchmarks, π^4 achieves the best overall average rank in geometry evaluation, alongside competitive performance in single- and multiview 2D/3D point tracking and moving-object…
foundation model4D scene perceptionuncalibrated RGB observationsdense geometrycamera posesmetric scale2D/3D point tracksmoving-object masksgeometry branchgeometry-conditioned motion branch
- Upvotes
- 4
- Authors
- 7
- Published
- Nov 17, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.