Hi Sebastian, could you also add me?:)

Thanks to my great collaborators: Long Chen, Elahe Arani and Oleg Sinavski

And thanks to Wayve for the great time during my internship and all the support.

Check out the paper or full video for more details.

Full Video: www.youtube.com/watch?v=Mpbn...

⛳️We introduce a DREAMING flag with which the model can differentiate between driving mode, where only safe instructions are executed, and dreaming mode, where the actions for all instructions are predicted.

💭Action Dreaming: A safe way to test Language-Action alignment. We test not only expert behaviour but a wide variety of possible actions (e.g., speed changes, driving towards a specific object, lane change manoeuvres).

🫱🏻‍🫲🏽 Language-Action Alignment: On normal driving datasets, the action can often be inferred from the visual cue alone. Our new dataset includes multiple different actions for each sample, together with the language instruction. This forces the model to listen to the instruction.

🥇State-of-the-art: SimLingo is the first VLA model on the CARLA Leaderboard, achieving state-of-the-art driving performance on multiple benchmarks.