Compact & scalable: 469M params, 280h video. SOTA metrics and realistic trajectories in tough urban/turning scenes. #WorldModels #AutonomousDriving #GenerativeAI

Sample 3:

Sample 2:

Orbis can generate realistic and diverse scenes from just 5 context frames.
Sample 1:

Orbis enables realistic long-horizon rollouts.

The paper highlights the limitations of state-of-the-art video generation models and shows that Orbis more closely tracks the original trajectory curvature and speed, while also generating novel content beyond the unseen horizon.

We train an autoregressive two-stage world model. The hybrid tokenizer is trained with both continuous and discrete objectives to handle both representations. Subsequently, we compare continuous and discrete world models under the same representation. we show the advantage of continuous one.

We highlight shortcomings of current driving world models by introducing a new metric based on trajectory evaluation. Furthermore, we question whether a discrete or continuous latent space leads to better long-horizon rollouts.