Mahi Shafiullah
@notmahi.bsky.social
Building generally intelligent robots that *just work* everywhere, out of the box, at Berkeley AI Research (BAIR) and Meta FAIR.
Previously at NYU Courant, MIT and visiting researcher at Meta AI.
https://mahis.life/
Previously at NYU Courant, MIT and visiting researcher at Meta AI.
https://mahis.life/
Reposted by Mahi Shafiullah
Ever struggled with multi-sensor data from cameras, depth sensors, and other custom sensors? Meet AnySense—an iPhone app for effortless data acquisition and streaming. Working with multimodal sensor data will never be a chore again!
February 26, 2025 at 4:49 PM
Ever struggled with multi-sensor data from cameras, depth sensors, and other custom sensors? Meet AnySense—an iPhone app for effortless data acquisition and streaming. Working with multimodal sensor data will never be a chore again!
Reposted by Mahi Shafiullah
We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream:
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
February 26, 2025 at 3:14 PM
We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream:
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
Reposted by Mahi Shafiullah
Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)
February 20, 2025 at 10:01 PM
Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)
Reposted by Mahi Shafiullah
One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies
February 20, 2025 at 6:17 PM
One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies
Reposted by Mahi Shafiullah
Can we extend the power of world models beyond just online model-based learning? Absolutely!
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
January 31, 2025 at 7:24 PM
Can we extend the power of world models beyond just online model-based learning? Absolutely!
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
Reposted by Mahi Shafiullah
My advisor warned me that academics trend towards bitterness. He encouraged me to intentionally resist this, remember where I came from, and never forget the privilege of getting to spend a life working with knowledge and ideas. He too said that bitterness and resentment is easy.
In high school, my social studies teacher took me aside and told me that "cynicism was the easy path" and that I needed to resist it. It's probably the most important thing I learned in school.
Happy New Year. 11/11
January 4, 2025 at 4:17 PM
My advisor warned me that academics trend towards bitterness. He encouraged me to intentionally resist this, remember where I came from, and never forget the privilege of getting to spend a life working with knowledge and ideas. He too said that bitterness and resentment is easy.
Reposted by Mahi Shafiullah
New paper! We show that by using keypoint-based image representation, robot policies become robust to different object types and background changes.
We call this method Prescriptive Point Priors for robot Policies or P3-PO in short. Full project is here: point-priors.github.io
We call this method Prescriptive Point Priors for robot Policies or P3-PO in short. Full project is here: point-priors.github.io
December 10, 2024 at 8:32 PM
New paper! We show that by using keypoint-based image representation, robot policies become robust to different object types and background changes.
We call this method Prescriptive Point Priors for robot Policies or P3-PO in short. Full project is here: point-priors.github.io
We call this method Prescriptive Point Priors for robot Policies or P3-PO in short. Full project is here: point-priors.github.io
Reposted by Mahi Shafiullah
Modern policy architectures are unnecessarily complex. In our #NeurIPS2024 project called BAKU, we focus on what really matters for good policy learning.
BAKU is modular, language-conditioned, compatible with multiple sensor streams & action multi-modality, and importantly fully open-source!
BAKU is modular, language-conditioned, compatible with multiple sensor streams & action multi-modality, and importantly fully open-source!
December 9, 2024 at 11:33 PM
Modern policy architectures are unnecessarily complex. In our #NeurIPS2024 project called BAKU, we focus on what really matters for good policy learning.
BAKU is modular, language-conditioned, compatible with multiple sensor streams & action multi-modality, and importantly fully open-source!
BAKU is modular, language-conditioned, compatible with multiple sensor streams & action multi-modality, and importantly fully open-source!
Reposted by Mahi Shafiullah
Since we are nearing the end of the year, I'll revisit some of our work I'm most excited about from the last year and maybe a sneak peek of what we are up to next.
To start of, Robot Utility Models, which enables zero-shot deployment. In the video below, the robot hasnt seen these doors before.
To start of, Robot Utility Models, which enables zero-shot deployment. In the video below, the robot hasnt seen these doors before.
December 8, 2024 at 2:32 AM
Since we are nearing the end of the year, I'll revisit some of our work I'm most excited about from the last year and maybe a sneak peek of what we are up to next.
To start of, Robot Utility Models, which enables zero-shot deployment. In the video below, the robot hasnt seen these doors before.
To start of, Robot Utility Models, which enables zero-shot deployment. In the video below, the robot hasnt seen these doors before.
Reposted by Mahi Shafiullah
I'd like to introduce what I've been working at @hellorobot.bsky.social: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration.
Check it out: github.com/hello-robot/...
Thread ->
Check it out: github.com/hello-robot/...
Thread ->
December 3, 2024 at 4:51 PM
I'd like to introduce what I've been working at @hellorobot.bsky.social: Stretch AI, a set of open-source tools for language-guided autonomy, exploration, navigation, and learning from demonstration.
Check it out: github.com/hello-robot/...
Thread ->
Check it out: github.com/hello-robot/...
Thread ->
Reposted by Mahi Shafiullah
Turns out aria-glasses are a very useful tool to demonstrate actions to robots: Based on egocentric video we track dynamic changes in a scene graph and use the representation to replay or plan interactions for robots
🔗 behretj.github.io/LostAndFound/
📄 arxiv.org/abs/2411.19162
📺 youtu.be/xxMsaBSeMXo
🔗 behretj.github.io/LostAndFound/
📄 arxiv.org/abs/2411.19162
📺 youtu.be/xxMsaBSeMXo
December 3, 2024 at 12:01 PM
Turns out aria-glasses are a very useful tool to demonstrate actions to robots: Based on egocentric video we track dynamic changes in a scene graph and use the representation to replay or plan interactions for robots
🔗 behretj.github.io/LostAndFound/
📄 arxiv.org/abs/2411.19162
📺 youtu.be/xxMsaBSeMXo
🔗 behretj.github.io/LostAndFound/
📄 arxiv.org/abs/2411.19162
📺 youtu.be/xxMsaBSeMXo
Reposted by Mahi Shafiullah
A reminder for folks in financial need: many PhD applications have application fee waivers, those waivers are not super onerous, and they are usually granted (at least at the two schools I'm familiar with). Please take advantage of them.
December 1, 2024 at 4:49 PM
A reminder for folks in financial need: many PhD applications have application fee waivers, those waivers are not super onerous, and they are usually granted (at least at the two schools I'm familiar with). Please take advantage of them.
Reposted by Mahi Shafiullah
I collected some folk knowledge for RL and stuck them in my lecture slides a couple weeks back: web.mit.edu/6.7920/www/l... See Appendix B... sorry, I know, appendix of a lecture slide deck is not the best for discovery. Suggestions very welcome.
web.mit.edu
November 27, 2024 at 1:36 PM
I collected some folk knowledge for RL and stuck them in my lecture slides a couple weeks back: web.mit.edu/6.7920/www/l... See Appendix B... sorry, I know, appendix of a lecture slide deck is not the best for discovery. Suggestions very welcome.
Reposted by Mahi Shafiullah
On one of the first projects I supervised in my PhD, a student repeatedly ignored suggestions to commit and then accidentally deleted the project at the end of the semester. Please use git! There are even "fun" games you can use to learn it:
learngitbranching.js.org
learngitbranching.js.org
Learn Git Branching
An interactive Git visualization tool to educate and challenge!
learngitbranching.js.org
November 27, 2024 at 3:40 PM
On one of the first projects I supervised in my PhD, a student repeatedly ignored suggestions to commit and then accidentally deleted the project at the end of the semester. Please use git! There are even "fun" games you can use to learn it:
learngitbranching.js.org
learngitbranching.js.org
Reposted by Mahi Shafiullah
Interesting article but the author drank the Kool-Aid and never sought out other viewpoints: “Foundation models like GPT-4 have largely subsumed [previous] models that help robots with planning and vision, and locomotion and dexterity will probably soon be subsumed, too.”
November 26, 2024 at 4:40 PM
Interesting article but the author drank the Kool-Aid and never sought out other viewpoints: “Foundation models like GPT-4 have largely subsumed [previous] models that help robots with planning and vision, and locomotion and dexterity will probably soon be subsumed, too.”
Reposted by Mahi Shafiullah
I'll be presenting AnySkin at the Stanford Center for Design Research today at 2pm! Stop by for a chat and try the sensor out!
More info: any-skin.github.io
More info: any-skin.github.io
November 25, 2024 at 6:15 PM
I'll be presenting AnySkin at the Stanford Center for Design Research today at 2pm! Stop by for a chat and try the sensor out!
More info: any-skin.github.io
More info: any-skin.github.io
Reposted by Mahi Shafiullah
Reposted by Mahi Shafiullah
A reminder that many feeds here are non algorithmic so reposting is more helpful than it is on twitter
November 25, 2024 at 3:59 PM
A reminder that many feeds here are non algorithmic so reposting is more helpful than it is on twitter
Reposted by Mahi Shafiullah
This week's #PaperILike is "Robots for Humanity: In-Home Deployment of Stretch RE2" (Ranganeni et al., HRI 2024).
This is probably the most inspiring robot video/demo that I've ever seen.
Video: www.youtube.com/watch?v=K2U7...
Paper: dl.acm.org/doi/abs/10.1...
This is probably the most inspiring robot video/demo that I've ever seen.
Video: www.youtube.com/watch?v=K2U7...
Paper: dl.acm.org/doi/abs/10.1...
Robots for Humanity: In-Home Deployment of Stretch RE2
YouTube video by Vinitha Ranganeni
www.youtube.com
November 24, 2024 at 5:55 PM
This week's #PaperILike is "Robots for Humanity: In-Home Deployment of Stretch RE2" (Ranganeni et al., HRI 2024).
This is probably the most inspiring robot video/demo that I've ever seen.
Video: www.youtube.com/watch?v=K2U7...
Paper: dl.acm.org/doi/abs/10.1...
This is probably the most inspiring robot video/demo that I've ever seen.
Video: www.youtube.com/watch?v=K2U7...
Paper: dl.acm.org/doi/abs/10.1...
Reposted by Mahi Shafiullah
PSA: Can we use more bar charts in ML papers?
I can't recall the last time I wanted to compare dozens of numbers in a table to two decimal places. A visualization makes it much clearer whether claimed differences are significant.
I can't recall the last time I wanted to compare dozens of numbers in a table to two decimal places. A visualization makes it much clearer whether claimed differences are significant.
November 24, 2024 at 9:06 AM
PSA: Can we use more bar charts in ML papers?
I can't recall the last time I wanted to compare dozens of numbers in a table to two decimal places. A visualization makes it much clearer whether claimed differences are significant.
I can't recall the last time I wanted to compare dozens of numbers in a table to two decimal places. A visualization makes it much clearer whether claimed differences are significant.
Reposted by Mahi Shafiullah
Hi, I am new here and would love to interact with folks interested in robotics & AI.
A bit about myself: I currently run a robotics lab at NYU, where we are building general-purpose robots. Some of our latest projects are on www.lerrelpinto.com.
If you have questions ask away!
A bit about myself: I currently run a robotics lab at NYU, where we are building general-purpose robots. Some of our latest projects are on www.lerrelpinto.com.
If you have questions ask away!
November 14, 2024 at 4:40 AM
Hi, I am new here and would love to interact with folks interested in robotics & AI.
A bit about myself: I currently run a robotics lab at NYU, where we are building general-purpose robots. Some of our latest projects are on www.lerrelpinto.com.
If you have questions ask away!
A bit about myself: I currently run a robotics lab at NYU, where we are building general-purpose robots. Some of our latest projects are on www.lerrelpinto.com.
If you have questions ask away!
Reposted by Mahi Shafiullah
In Robot Utility Models, we collected a large number of demonstration videos — 1000 examples in 40 environments — using a tool called “the Stick." The lead author even demonstrated it live at CoRL
on completely new furniture. This one really does 100% actually work - I’ve used it in my house.
on completely new furniture. This one really does 100% actually work - I’ve used it in my house.
November 14, 2024 at 2:22 AM
In Robot Utility Models, we collected a large number of demonstration videos — 1000 examples in 40 environments — using a tool called “the Stick." The lead author even demonstrated it live at CoRL
on completely new furniture. This one really does 100% actually work - I’ve used it in my house.
on completely new furniture. This one really does 100% actually work - I’ve used it in my house.
Reposted by Mahi Shafiullah
Very excited about this new project, DynaMem. It allows our robots to function in previously unseen environments, performing long-horizon manipulation tasks. Most importantly it *generalizes*, meaning you can try it out on a wide variety of homes and on different objects. (4x video)
November 9, 2024 at 3:26 PM
Very excited about this new project, DynaMem. It allows our robots to function in previously unseen environments, performing long-horizon manipulation tasks. Most importantly it *generalizes*, meaning you can try it out on a wide variety of homes and on different objects. (4x video)