More details and results in the paper, and stay tuned for the 2HANDS dataset & code release!

📄Paper: arxiv.org/abs/2503.09320
🌐 Website: sites.google.com/view/2handedafforder

Work done with Marin Heidinger, Vignesh Prasad & @georgiachal.bsky.social

See you in Hawaii at #ICCV2025! 🌴

We can then use our high-quality dataset to train or fine-tune a VLM that takes in the activity/task text prompt as input and predicts bimanual affordance masks (for a left and right robot hand)

4/5

We extract bimanual affordance masks from egocentric RGB video datasets using video-based hand inpainting and object reconstruction.

No manual labeling is required. The narrations from egocentric datasets also provide free-form text supervision! (Eg. "pour milk into bowl")

3/5

The Problem:
Most affordance detection methods just segment object parts & do not predict actionable regions for robots!

Our solution?
Use egocentric bimanual human videos to extract precise affordance regions considering object relationships, context, & hand coordination!

2/5

Learn more at the workshop website: egoact.github.io/rss2025

Happy to be organizing this with @georgiachal.bsky.social, Yu Xiang, @danfei.bsky.social and @galasso.bsky.social!

Call for Contributions:
We’re inviting contributions in the form of:
📝 Full papers OR
📝 4-page extended abstracts
🗓️ Submission Deadline: April 30, 2025
🏆 Best Paper Award, sponsored by Meta!

Core workshop topics include:
🥽 Egocentric interfaces for robot learning
🧠 High-level action & scene understanding
🤝 Human-to-robot transfer
🧱 Foundation models from human activity datasets
🛠️ Egocentric world models for high-level planning & low-level manipulation

I'm working on Robot learning and perception : )