Johannes Schusterbauer
@joh-schb.bsky.social
PhD Student @ CompVis group, LMU Munich
Working on diffusion & flow models🫶
Working on diffusion & flow models🫶
Pinned
🤔 What if you could generate an entire image using just one continuous token?
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
Reposted by Johannes Schusterbauer
I’m thrilled to share that I’ll present two first-authored papers at #ICCV2025 🌺 in Honolulu together with @mgui7.bsky.social ! 🏝️
(Thread 🧵👇)
(Thread 🧵👇)
October 18, 2025 at 3:01 AM
I’m thrilled to share that I’ll present two first-authored papers at #ICCV2025 🌺 in Honolulu together with @mgui7.bsky.social ! 🏝️
(Thread 🧵👇)
(Thread 🧵👇)
🤔 What if you could generate an entire image using just one continuous token?
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
October 17, 2025 at 10:21 AM
🤔 What if you could generate an entire image using just one continuous token?
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
💡 It works if we leverage a self-supervised representation!
Meet RepTok🦎: A generative model that encodes an image into a single continuous latent while keeping realism and semantics. 🧵 👇
Reposted by Johannes Schusterbauer
🤔 What happens when you poke a scene — and your model has to predict how the world moves in response?
We built the Flow Poke Transformer (FPT) to model multi-modal scene dynamics from sparse interactions.
It learns to predict the 𝘥𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘰𝘯 of motion itself 🧵👇
We built the Flow Poke Transformer (FPT) to model multi-modal scene dynamics from sparse interactions.
It learns to predict the 𝘥𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘰𝘯 of motion itself 🧵👇
October 15, 2025 at 1:56 AM
🤔 What happens when you poke a scene — and your model has to predict how the world moves in response?
We built the Flow Poke Transformer (FPT) to model multi-modal scene dynamics from sparse interactions.
It learns to predict the 𝘥𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘰𝘯 of motion itself 🧵👇
We built the Flow Poke Transformer (FPT) to model multi-modal scene dynamics from sparse interactions.
It learns to predict the 𝘥𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘪𝘰𝘯 of motion itself 🧵👇
Looking forward to attending #CVPR2025 in Nashville next week 🎸🎶 @mgui7.bsky.social and I will be presenting our latest work:
🌊 Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
🌊 Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
June 6, 2025 at 3:48 PM
Looking forward to attending #CVPR2025 in Nashville next week 🎸🎶 @mgui7.bsky.social and I will be presenting our latest work:
🌊 Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
🌊 Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment
Sunrise in the office after the #ICCV deadline night with @mgui7.bsky.social 🚀
March 8, 2025 at 5:46 AM
Sunrise in the office after the #ICCV deadline night with @mgui7.bsky.social 🚀
Reposted by Johannes Schusterbauer
Reposted by Johannes Schusterbauer
Over 60 German universities and research institutions announced their departure from X today.
Starkes Signal!!
Über 60 dt. Hochschulen & Forschungsinstitutionen haben heute ihren Ausstieg bei X bekanntgegeben, s.u. #eXit
X sei nicht mehr vereinbar mit ihren Grundwerten: „Weltoffenheit, wissenschaftliche Integrität, Transparenz und demokratischer Diskurs.“
Liste der Beteiligten hier:
Über 60 dt. Hochschulen & Forschungsinstitutionen haben heute ihren Ausstieg bei X bekanntgegeben, s.u. #eXit
X sei nicht mehr vereinbar mit ihren Grundwerten: „Weltoffenheit, wissenschaftliche Integrität, Transparenz und demokratischer Diskurs.“
Liste der Beteiligten hier:
Hochschulen und Forschungsinstitutionen verlassen Plattform X - Gemeinsam für Vielfalt, Freiheit und Wissenschaft
nachrichten.idw-online.de
January 10, 2025 at 10:45 AM
Over 60 German universities and research institutions announced their departure from X today.
Reposted by Johannes Schusterbauer
🤔When combining Vision-language models (VLMs) with Large language models (LLMs), do VLMs benefit from additional genuine semantics or artificial augmentations of the text for downstream tasks?
🤨Interested? Check out our latest work at #AAAI25:
💻Code and 📝Paper at: github.com/CompVis/DisCLIP
🧵👇
🤨Interested? Check out our latest work at #AAAI25:
💻Code and 📝Paper at: github.com/CompVis/DisCLIP
🧵👇
January 8, 2025 at 3:54 PM
🤔When combining Vision-language models (VLMs) with Large language models (LLMs), do VLMs benefit from additional genuine semantics or artificial augmentations of the text for downstream tasks?
🤨Interested? Check out our latest work at #AAAI25:
💻Code and 📝Paper at: github.com/CompVis/DisCLIP
🧵👇
🤨Interested? Check out our latest work at #AAAI25:
💻Code and 📝Paper at: github.com/CompVis/DisCLIP
🧵👇
Congrats to @frankfundel.bsky.social for publishing this work at WACV🔥
Has been a pleasure to jointly work on this topic with such a talented master student🤗
Looking forward to seeing what comes next!🚀
Has been a pleasure to jointly work on this topic with such a talented master student🤗
Looking forward to seeing what comes next!🚀
Did you know you can distill the capabilities of a large diffusion model into a small ViT? ⚗️
We showed exactly that for a fundamental task:
semantic correspondence📍
A thread 🧵👇
We showed exactly that for a fundamental task:
semantic correspondence📍
A thread 🧵👇
December 6, 2024 at 5:05 PM
Congrats to @frankfundel.bsky.social for publishing this work at WACV🔥
Has been a pleasure to jointly work on this topic with such a talented master student🤗
Looking forward to seeing what comes next!🚀
Has been a pleasure to jointly work on this topic with such a talented master student🤗
Looking forward to seeing what comes next!🚀
Awesome work from some colleagues cleaning up diffusion features!🚀
🤔 Why do we extract diffusion features from noisy images? Isn’t that destroying information?
Yes, it is - but we found a way to do better. 🚀
Here’s how we unlock better features, no noise, no hassle.
📝 Project Page: compvis.github.io/cleandift
💻 Code: github.com/CompVis/clea...
🧵👇
Yes, it is - but we found a way to do better. 🚀
Here’s how we unlock better features, no noise, no hassle.
📝 Project Page: compvis.github.io/cleandift
💻 Code: github.com/CompVis/clea...
🧵👇
December 5, 2024 at 7:06 AM
Awesome work from some colleagues cleaning up diffusion features!🚀
Reposted by Johannes Schusterbauer
IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.
Taming Transformers for High-Resolution Image Synthesis
Designed to learn long-range interactions on sequential data, transformers continue to show state-of-the-art results on a wide variety of tasks. In contrast to CNNs, they contain no inductive bias tha...
arxiv.org
November 28, 2024 at 12:09 AM
IMO VQGAN is why GANs deserve the NeurIPS test of time award. Suddenly our image representations were an order of magnitude more compact. Absolute game changer for generative modelling at scale, and the basis for latent diffusion models.
Reposted by Johannes Schusterbauer
Check out my GenAI starter pack! go.bsky.app/BT1bRvZ
November 23, 2024 at 10:45 AM
Check out my GenAI starter pack! go.bsky.app/BT1bRvZ
Reposted by Johannes Schusterbauer
After many years, our lab finally has a social media presence at @compvis.bsky.social ! 🥳
Give it a follow, we have some amazing research on generative computer vision coming soon!
Give it a follow, we have some amazing research on generative computer vision coming soon!
November 20, 2024 at 6:31 PM
After many years, our lab finally has a social media presence at @compvis.bsky.social ! 🥳
Give it a follow, we have some amazing research on generative computer vision coming soon!
Give it a follow, we have some amazing research on generative computer vision coming soon!
Reposted by Johannes Schusterbauer
Reposted by Johannes Schusterbauer
In a gratuitous attempt to acquire more followers myself 😁, I've made a start on a "starter pack". Hopefully as more people from 🐦 make it over to 🦋, we can extend this a bit. Suggestions welcome!
I've noticed not all accounts seem to be eligible to be added, anyone know what's up with that? 🤔
I've noticed not all accounts seem to be eligible to be added, anyone know what's up with that? 🤔
November 15, 2024 at 8:04 PM
In a gratuitous attempt to acquire more followers myself 😁, I've made a start on a "starter pack". Hopefully as more people from 🐦 make it over to 🦋, we can extend this a bit. Suggestions welcome!
I've noticed not all accounts seem to be eligible to be added, anyone know what's up with that? 🤔
I've noticed not all accounts seem to be eligible to be added, anyone know what's up with that? 🤔