Sean Man
@sean-man.bsky.social
Ph.D. student Technion under Prof. Michael Elad ; Researching Image Inverse problems ; sean_8100🐦
✨ The result?
✅ Sharper images
✅ Significant speedups
✅ A simple framework for inverse problems with latent diffusion priors.
🧵[6/7]
✅ Sharper images
✅ Significant speedups
✅ A simple framework for inverse problems with latent diffusion priors.
🧵[6/7]
January 22, 2025 at 5:27 PM
✨ The result?
✅ Sharper images
✅ Significant speedups
✅ A simple framework for inverse problems with latent diffusion priors.
🧵[6/7]
✅ Sharper images
✅ Significant speedups
✅ A simple framework for inverse problems with latent diffusion priors.
🧵[6/7]
🔥 Our solution: What if we could bypass the decoder entirely?
We designed a latent operator that mimics image-space degradations directly in the latent space, eliminating the use of the decoder and its Jacobian.
🧵[5/7]
We designed a latent operator that mimics image-space degradations directly in the latent space, eliminating the use of the decoder and its Jacobian.
🧵[5/7]
January 22, 2025 at 5:27 PM
🔥 Our solution: What if we could bypass the decoder entirely?
We designed a latent operator that mimics image-space degradations directly in the latent space, eliminating the use of the decoder and its Jacobian.
🧵[5/7]
We designed a latent operator that mimics image-space degradations directly in the latent space, eliminating the use of the decoder and its Jacobian.
🧵[5/7]
⚠️ Worse, backpropagating through the decoder introduces artifacts into the restored images due to its Jacobian.
🧵[4/7]
🧵[4/7]
January 22, 2025 at 5:27 PM
⚠️ Worse, backpropagating through the decoder introduces artifacts into the restored images due to its Jacobian.
🧵[4/7]
🧵[4/7]
💡 The challenge: Solving inverse problems with latent diffusion models is tricky because degradation operators (e.g., blur, noise) are defined in image space.
This forces costly decoding steps at every iteration, slowing everything down.
🧵[3/7]
This forces costly decoding steps at every iteration, slowing everything down.
🧵[3/7]
January 22, 2025 at 5:27 PM
💡 The challenge: Solving inverse problems with latent diffusion models is tricky because degradation operators (e.g., blur, noise) are defined in image space.
This forces costly decoding steps at every iteration, slowing everything down.
🧵[3/7]
This forces costly decoding steps at every iteration, slowing everything down.
🧵[3/7]
🚀 Excited to share our latest research: “SILO: Solving Inverse Problems with Latent Operators”!
A surprisingly simple approach to image restoration with latent diffusion models that achieves SOTA results while being 2.5x–10x faster than prior methods.
🧵[1/7]
A surprisingly simple approach to image restoration with latent diffusion models that achieves SOTA results while being 2.5x–10x faster than prior methods.
🧵[1/7]
January 22, 2025 at 5:27 PM
🚀 Excited to share our latest research: “SILO: Solving Inverse Problems with Latent Operators”!
A surprisingly simple approach to image restoration with latent diffusion models that achieves SOTA results while being 2.5x–10x faster than prior methods.
🧵[1/7]
A surprisingly simple approach to image restoration with latent diffusion models that achieves SOTA results while being 2.5x–10x faster than prior methods.
🧵[1/7]