This work wouldn't have been possible without my internship mentor Kai, advisors @snavely.bsky.social @bharathhariharan.bsky.social, and other coauthors.

Project Page: genechou.com/kfcw
N/N

Our method outperforms commercial models in terms of geometric and appearance consistency, and we show that video models trained with 3D-aware objectives can be useful as 3D priors for downstream tasks such as SfM and 3DGS. Check out our paper for more details! 5/N

We design a self-supervised method that takes advantage of the consistency of videos and variability of multiview internet photos to train a 3D-aware video model without ANY 3D annotations. We name our method KFC-W (KeyFrame-Conditioned video generation in-the-Wild). 4/N

This task is difficult for video models! They perform visually pleasing morphing but can ignore scene identity and hallucinate new structures. Our main insight is that training for general video synthesis is not enough: we need to introduce scalable, 3D-aware objectives. 3/N

We propose the task of generating videos from sparse (2-5), unposed internet photos. A model’s ability to capture underlying geometry, recognize scene identity, and relate frames in terms of camera position reflects its understanding of 3D structure and scene layout. 2/N