Katrin Renz
banner
katrinrenz.bsky.social
Katrin Renz
@katrinrenz.bsky.social
https://katrinrenz.de/
LLMs + Autonomous Driving.
PhD Student at Uni Tübingen with Andreas Geiger.
Previously at Wayve & Uni Oxford, VGG.
Hi Sebastian, could you also add me?:)
May 9, 2025 at 5:34 AM
Thanks to my great collaborators: Long Chen, Elahe Arani and Oleg Sinavski

And thanks to Wayve for the great time during my internship and all the support.
May 8, 2025 at 3:25 PM
Check out the paper or full video for more details.

Full Video: www.youtube.com/watch?v=Mpbn...
[CVPR25, Spotl.] SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignment
YouTube video by Katrin Renz
www.youtube.com
May 8, 2025 at 3:25 PM
⛳️We introduce a DREAMING flag with which the model can differentiate between driving mode, where only safe instructions are executed, and dreaming mode, where the actions for all instructions are predicted.
May 8, 2025 at 3:25 PM
💭Action Dreaming: A safe way to test Language-Action alignment. We test not only expert behaviour but a wide variety of possible actions (e.g., speed changes, driving towards a specific object, lane change manoeuvres).
May 8, 2025 at 3:25 PM
🫱🏻‍🫲🏽 Language-Action Alignment: On normal driving datasets, the action can often be inferred from the visual cue alone. Our new dataset includes multiple different actions for each sample, together with the language instruction. This forces the model to listen to the instruction.
May 8, 2025 at 3:25 PM
🥇State-of-the-art: SimLingo is the first VLA model on the CARLA Leaderboard, achieving state-of-the-art driving performance on multiple benchmarks.
May 8, 2025 at 3:25 PM