Lerrel Pinto
@lerrelpinto.com
Assistant Professor of CS @nyuniversity.
I like robots!
I like robots!
Pinned
Lerrel Pinto
@lerrelpinto.com
· Apr 18
We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced!
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
Reposted by Lerrel Pinto
How is AI helping robots to generalise their skills to unfamiliar environments? 🤖 🏠
In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.
Available wherever you get your podcasts: linktr.ee/robottalkpod
In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.
Available wherever you get your podcasts: linktr.ee/robottalkpod
May 21, 2025 at 8:37 AM
How is AI helping robots to generalise their skills to unfamiliar environments? 🤖 🏠
In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.
Available wherever you get your podcasts: linktr.ee/robottalkpod
In the latest episode, I chatted to Prof. Lerrel Pinto (@lerrelpinto.com) from New York University about #robot learning and decision making.
Available wherever you get your podcasts: linktr.ee/robottalkpod
We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced!
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
April 18, 2025 at 6:53 PM
We just released RUKA, a $1300 humanoid hand that is 3D-printable, strong, precise, and fully open sourced!
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
The key technical breakthrough here is that we can control joints and fingertips of the robot **without joint encoders**. All we need here is self-supervised data collection and learning.
March 28, 2025 at 4:02 AM
Reposted by Lerrel Pinto
What would you love to know about #robot learning and decision making?
Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.
Send me your questions for Lerrel: robottalk.org/ask-a-question/
Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.
Send me your questions for Lerrel: robottalk.org/ask-a-question/
March 18, 2025 at 10:11 AM
What would you love to know about #robot learning and decision making?
Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.
Send me your questions for Lerrel: robottalk.org/ask-a-question/
Later this season, I'll be chatting to Prof. Lerrel Pinto (@lerrelpinto.com) from NYU about using machine learning to train robots to adapt to new environments.
Send me your questions for Lerrel: robottalk.org/ask-a-question/
Is there a word for the feeling when you want to cheer for the other team?
March 2, 2025 at 9:23 PM
Is there a word for the feeling when you want to cheer for the other team?
Reposted by Lerrel Pinto
Times Square
March 2, 2025 at 1:52 AM
Times Square
Reposted by Lerrel Pinto
Slava Ukraini 🇺🇦
February 28, 2025 at 6:20 PM
Slava Ukraini 🇺🇦
The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!
February 28, 2025 at 7:09 PM
The robot behaviors shown below are trained without any teleop, sim2real, genai, or motion planning. Simply show the robot a few examples of doing the task yourself, and our new method, called Point Policy, spits out a robot-compatible policy!
Reposted by Lerrel Pinto
It finally dawned on me. I couldn't figure out what was going in Washington, plus all the hype about humanoids, including the new genre of "humanoid theater" videos, unmoored from reality. Then I realized, we must indeed be living in a simulation. Scripted by an LLM. Our world is a confabulation!
February 27, 2025 at 8:24 PM
It finally dawned on me. I couldn't figure out what was going in Washington, plus all the hype about humanoids, including the new genre of "humanoid theater" videos, unmoored from reality. Then I realized, we must indeed be living in a simulation. Scripted by an LLM. Our world is a confabulation!
Reposted by Lerrel Pinto
This is important because the humble iPhone is one of the best accessories for embodied AI out there, if not actually the best. It's got a depth sensor, good camera, built-in internet, decent compute, and -- uniquely -- it has really good slam already built in.
February 26, 2025 at 4:20 PM
This is important because the humble iPhone is one of the best accessories for embodied AI out there, if not actually the best. It's got a depth sensor, good camera, built-in internet, decent compute, and -- uniquely -- it has really good slam already built in.
We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream:
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
February 26, 2025 at 3:14 PM
We just released AnySense, an iPhone app for effortless data acquisition and streaming for robotics. We leverage Apple’s development frameworks to record and stream:
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
1. RGBD + Pose data
2. Audio from the mic or custom contact microphones
3. Seamless Bluetooth integration for external sensors
Reposted by Lerrel Pinto
A useful “productivity” trick is to remind yourself that research should be fun and inspiring and if it’s not that something should change.
February 23, 2025 at 6:49 PM
A useful “productivity” trick is to remind yourself that research should be fun and inspiring and if it’s not that something should change.
Reposted by Lerrel Pinto
And I am an ally. If you are too, let the world know.
February 22, 2025 at 10:14 PM
And I am an ally. If you are too, let the world know.
Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)
February 20, 2025 at 10:01 PM
Just found a new winner for the most hype-baiting, unscientific plot I have seen. (From the recent Figure AI release)
Reposted by Lerrel Pinto
One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies
February 20, 2025 at 6:17 PM
One reason to be intolerant of misleading hype in tech and science is that tolerating the small lies and deception is how you get tolerance of big lies
Thank you to @sloanfoundation.bsky.social for this generous award to our lab. Hopefully this will bring us closer to building truly general-purpose robots!
🎉Congrats to the 126 early-career scientists who have been awarded a Sloan Research Fellowship this year! These exceptional scholars are drawn from 51 institutions across the US and Canada, and represent the next generation of groundbreaking researchers. sloan.org/fellowships/...
February 18, 2025 at 4:50 PM
Thank you to @sloanfoundation.bsky.social for this generous award to our lab. Hopefully this will bring us closer to building truly general-purpose robots!
Reposted by Lerrel Pinto
yoinked this image from Twitter, then ordered my copy
join me
join me
February 16, 2025 at 3:13 AM
yoinked this image from Twitter, then ordered my copy
join me
join me
A fun, clever idea from @upiter.bsky.social : treat code generation as a sequential editing problem -- this gives you loads of training data from synthetically editing existing code
And it works! Higher performance on HumanEval, MBPP, and CodeContests across small LMs like Gemma-2, Phi-3, Llama 3.1
And it works! Higher performance on HumanEval, MBPP, and CodeContests across small LMs like Gemma-2, Phi-3, Llama 3.1
Our paper showing that LMs benefit from human-like abstractions for code synthesis was accepted to ICLR! 🇸🇬
We show that order matters in code gen. -- casting code synthesis as a sequential edit problem by preprocessing examples in SFT data improves LM test-time scaling laws
We show that order matters in code gen. -- casting code synthesis as a sequential edit problem by preprocessing examples in SFT data improves LM test-time scaling laws
February 13, 2025 at 3:42 PM
A fun, clever idea from @upiter.bsky.social : treat code generation as a sequential editing problem -- this gives you loads of training data from synthetically editing existing code
And it works! Higher performance on HumanEval, MBPP, and CodeContests across small LMs like Gemma-2, Phi-3, Llama 3.1
And it works! Higher performance on HumanEval, MBPP, and CodeContests across small LMs like Gemma-2, Phi-3, Llama 3.1
We have been working a bunch on offline world models. Pre-trained features from DINOv2 seem really powerful for modeling. I hope this opens up a whole set of applications for decision making and robotics!
Check out the thread from @gaoyuezhou.bsky.social for more details.
Check out the thread from @gaoyuezhou.bsky.social for more details.
Can we extend the power of world models beyond just online model-based learning? Absolutely!
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
We believe the true potential of world models lies in enabling agents to reason at test time.
Introducing DINO-WM: World Models on Pre-trained Visual Features for Zero-shot Planning.
January 31, 2025 at 8:06 PM
We have been working a bunch on offline world models. Pre-trained features from DINOv2 seem really powerful for modeling. I hope this opens up a whole set of applications for decision making and robotics!
Check out the thread from @gaoyuezhou.bsky.social for more details.
Check out the thread from @gaoyuezhou.bsky.social for more details.
Reposted by Lerrel Pinto
To help people understand the importance of NIH, share what you’ve used their funding for (in easily understandable terms).
I’ll start: my NIH postdoc funding helped me develop and test AI tools that could identify skin cancer across diverse skin tones.
I’ll start: my NIH postdoc funding helped me develop and test AI tools that could identify skin cancer across diverse skin tones.
January 27, 2025 at 6:12 PM
To help people understand the importance of NIH, share what you’ve used their funding for (in easily understandable terms).
I’ll start: my NIH postdoc funding helped me develop and test AI tools that could identify skin cancer across diverse skin tones.
I’ll start: my NIH postdoc funding helped me develop and test AI tools that could identify skin cancer across diverse skin tones.
Reposted by Lerrel Pinto
If you’re in grad school, finding a therapist can be really helpful. The thing you’re doing is hard and it’s harder if you don’t have help managing imposter syndrome, stress, self esteem, and a whole bunch of other things.
January 9, 2025 at 3:20 AM
If you’re in grad school, finding a therapist can be really helpful. The thing you’re doing is hard and it’s harder if you don’t have help managing imposter syndrome, stress, self esteem, and a whole bunch of other things.
Reposted by Lerrel Pinto
omg a student somehow accidentally wrote an email addressed to a faculty-wide NYU listserv and my inbox is now a master class on who understands the difference between a listserv and an email chain
December 30, 2024 at 12:25 AM
omg a student somehow accidentally wrote an email addressed to a faculty-wide NYU listserv and my inbox is now a master class on who understands the difference between a listserv and an email chain
Reposted by Lerrel Pinto
Humans vs Ants: Problem-solving Skills
December 25, 2024 at 5:12 PM
Humans vs Ants: Problem-solving Skills
At NYU Abu Dhabi today and in love how cat friendly the campus is!
December 18, 2024 at 4:39 AM
At NYU Abu Dhabi today and in love how cat friendly the campus is!
Reposted by Lerrel Pinto
This holiday season, take a moment to visit your local bookstore. It’s about more than finding a great book—it’s about supporting the small businesses that keep our communities thriving.
December 15, 2024 at 12:41 AM
This holiday season, take a moment to visit your local bookstore. It’s about more than finding a great book—it’s about supporting the small businesses that keep our communities thriving.