Wrapping up my year as a Student Researcher at Google DeepMind today!🥲 It’s been an amazing experience (proof by picture😅).

Excited to join Waabi in the new year and do some cool research on robustness in 🚗 🤖!

I reflected on the past 3.5 years of my PhD today—here are some thoughts/

…and with that we get good quality masks, without human annotation or synthetic supervision!
Look at our website for more results!
romosfm.github.io

We train a tiny MLP that classifies SAMv2 features as moving or static given the weak supervisory signal from high and low error masks. These features help complete the motion masks over the video effectively!

so...how does it work?

We find the Fundamental matrix between each two adjacent frames in the video with RANSAC. We then identify parts of the frame that have a very low or a very high epipolar error, as weak supervision signals to find the moving objects. Now, how do we complete these signals?

Okay but why care about motion masks?
We show that good motion masks improve SfM performance, making COLMAP+our masks the SOTA on synthetic benchmarks. We also collect a real evaluation dataset with GT camera pose using a robotic arm, to evaluate our method in real casual captures.

Our masks are robust to slow or fast camera movements and can find multiple moving objects even when they are in the background. Look at the pedestrian! 🚶

First and foremost: some results!!
In RoMo an optimization process disentangles camera ego motion from scene motion, yielding masks for moving objects 🛵

Hello everyone!! 👋
Excited to be here and share our latest work to get started!

RoMo: Robust Motion Segmentation Improves Structure from Motion

romosfm.github.io

Boost the performance of your SfM pipeline on dynamic scenes! 🚀 RoMo masks dynamic objects in a video, in a zero-shot manner.