Excited to share our work at #CVPR2025!
👁️IRIS estimates accurate surface material, spatially-varying HDR lighting, and camera response function given a set of LDR images! It enables realistic, view-consistent, and controllable relighting and object insertion.
(links in 🧵)

The weather removal model successfully removes both transient (e.g., rain, snowflake) and persistent effects (e.g., puddle, snow cover), and can even restore sunny-day lighting from rainy/snowy videos.

WeatherWeaver combines two video diffusion models. The weather synthesis model generates realistic, temporally consistent weather, adapting shading naturally while preserving the original scene structure.

By combining and adjusting multiple weather effects, WeatherWeaver can simulate complex weather transitions, e.g. on 🌧️ rainy and ☃️ snowy days — without costly real-world acquisitions.

WeatherWeaver enables precise control of the weather effects by changing the intensity of the corresponding effects. 🌤️➡️🌥️

We train a video diffusion model to edit weather effects with precise control, using a novel data strategy combining synthetic videos, generative image editing, and auto-labeled real-world videos.

What if you could control the weather in any video — just like applying a filter?
Meet WeatherWeaver, a video model for controllable synthesis and removal of diverse weather effects — such as 🌧️ rain, ☃️ snow, 🌁 fog, and ☁️ clouds — for any input video.

🎬Imagine creating professional visual effects (VFX) with just words! We are excited to introduce AutoVFX, a framework that creates realistic video effects from natural language instructions!

This is a cool project led by Hao-Yu, and we will present it at #3DV 2025!

📢 Meet us at 3DV 2025 in Singapore!
We’re excited to present UrbanIR at 3DV 2025 @3dvconf.bsky.social , come chat with us to discuss future directions!

UrbanIR precisely controls the lighting, simulating different times of day without time-consuming tripod captures.

With estimated scene properties, UrbanIR integrates a physically-based shading model into neural field, rendering realistic videos from novel viewpoints and lighting conditions.

🔍 How does it work?
UrbanIR reconstructs scene properties—geometry, albedo, and shading—through inverse rendering. Since this is a highly ill-posed problem, we leverage 2D priors to guide the optimization.

✨What if we could transform a daytime driving video into a realistic nighttime scene—without ever stepping outside again?
We introduce UrbanIR, a neural rendering framework for 💡relighting, 🌃nighttime simulation, and 🚘 object insertion—all from a single video of urban scenes!