PS: 🌀 We recently released spatial-reasoners, a general toolkit to apply SRMs to a wide range of different domains: spatialreasoners.github.io

We find that model hallucination can be drastically reduced by choosing the right configuration, allowing to significantly increase performance in complex reasoning tasks like solving visual Sudoku.

Our Spatial Reasoning Models allow to explore the space between parallel and autoregressive diffusion models with different methods for choosing generation order.

Project Website: geometric-rl.mpi-inf.mpg.de/srm/

MET3R quantitatively measures 3D consistency between two images via DUSt3R reconstruction and feature comparison. It does not require camera poses.

Code is available for plug-and-play use. We also provide an open source multi-view latent diffusion model for further research!

Project page: geometric-rl.mpi-inf.mpg.de/met3r/

We also show that good orders can be predicted by uncertainty, which is crucial for the Sudoku task to be solved well.

Spatial Reasoning Models (SRMs) are a framework to propagate belief over a set of continuous variables (e.g. image patches) with generative denoising models.

It allows to explore the amount of (soft) sequentialization and the order of generation, both having significant impact on reasoning quality.

MET3R measures 3D consistency between two images without camera poses via DUSt3R reconstruction and feature comparison.

Code is available for plug-and-play use. We also provide an open source multi-view latent diffusion model for further research!

Project page: geometric-rl.mpi-inf.mpg.de/met3r/