Stone Tao
@stonet2000.bsky.social
PhDing @UCSanDiego @NVIDIA @hillbot_ai on scalable robot learning and embodied AI. Co-founded @LuxAIChallenge to build AI competitions. @NSF GRFP fellow
http://stoneztao.com
http://stoneztao.com
I will be in SF from November 10-17 🌉
If you work on something interesting and want to meet up let me know! Or if there’s a fun event i’d love some invites 😃
I’m also looking to visit robotics companies (especially startups!), if you have time lmk if I can visit!
If you work on something interesting and want to meet up let me know! Or if there’s a fun event i’d love some invites 😃
I’m also looking to visit robotics companies (especially startups!), if you have time lmk if I can visit!
November 1, 2025 at 6:45 AM
I will be in SF from November 10-17 🌉
If you work on something interesting and want to meet up let me know! Or if there’s a fun event i’d love some invites 😃
I’m also looking to visit robotics companies (especially startups!), if you have time lmk if I can visit!
If you work on something interesting and want to meet up let me know! Or if there’s a fun event i’d love some invites 😃
I’m also looking to visit robotics companies (especially startups!), if you have time lmk if I can visit!
1/1 neurips submission accepted! Will share more details later, covers an often overlooked aspect of on policy RL training when scaling to large scale parallelized environments
September 19, 2025 at 5:01 PM
1/1 neurips submission accepted! Will share more details later, covers an often overlooked aspect of on policy RL training when scaling to large scale parallelized environments
Reposted by Stone Tao
Rumors going around are that NeurIPS PCs had to reject ~400 papers because of venue constraints that were in the accept pile.
Hard things like this happen but the PCs and conference should explicitly tell all the authors who had this happen to them. Or create something like accept w/o presentation.
Hard things like this happen but the PCs and conference should explicitly tell all the authors who had this happen to them. Or create something like accept w/o presentation.
September 19, 2025 at 4:31 PM
Rumors going around are that NeurIPS PCs had to reject ~400 papers because of venue constraints that were in the accept pile.
Hard things like this happen but the PCs and conference should explicitly tell all the authors who had this happen to them. Or create something like accept w/o presentation.
Hard things like this happen but the PCs and conference should explicitly tell all the authors who had this happen to them. Or create something like accept w/o presentation.
my partner just hopped online, very talented with a great eye for design. She’s also behind all the beautiful graphics in the Lux AI Challenge that helped us gain 100s of competitors! She’s on the job market right now so DM her if you are looking for a designer/marketer/PM!
New to Bluesky, so here's my lil' intro:
Hi, I'm Isa 👋 part designer, part marketer, part product wrangler 🧩
Built Dora AI and more with the rad team at Dora (dora.run) and 3 seasons of the Lux AI Challenge as CDO 🤩
Here to meet fellow product enthusiasts!
Hi, I'm Isa 👋 part designer, part marketer, part product wrangler 🧩
Built Dora AI and more with the rad team at Dora (dora.run) and 3 seasons of the Lux AI Challenge as CDO 🤩
Here to meet fellow product enthusiasts!
September 18, 2025 at 4:56 AM
my partner just hopped online, very talented with a great eye for design. She’s also behind all the beautiful graphics in the Lux AI Challenge that helped us gain 100s of competitors! She’s on the job market right now so DM her if you are looking for a designer/marketer/PM!
Opensourcing a tool to calibrate camera extrinsics painlessly in a minute, no checkerboards! It's based on EasyHEC, using diff. rendering to optimize extrinsics given object meshes+poses. Crazy that even a piece of paper works too.
Code:
github.com/StoneT2000/s...
(paper example in next post)
Code:
github.com/StoneT2000/s...
(paper example in next post)
September 9, 2025 at 2:31 AM
Opensourcing a tool to calibrate camera extrinsics painlessly in a minute, no checkerboards! It's based on EasyHEC, using diff. rendering to optimize extrinsics given object meshes+poses. Crazy that even a piece of paper works too.
Code:
github.com/StoneT2000/s...
(paper example in next post)
Code:
github.com/StoneT2000/s...
(paper example in next post)
Big thanks to Jeremy Morgan (PhD @ USC) for helping massively accelerate the IK solvers in ManiSkill! Open source really helps grow this project (and i get to learn tons of new things like IK optimization)
github.com/haosulab/Man...
github.com/haosulab/Man...
[Feature] Faster IK based controllers using levenberg-marqurdt and align pd ee pos controllers in gpu/cpu sims by StoneT2000 · Pull Request #1213 · haosulab/ManiSkill
Cleaned up version of #1148 to close #955.
Updates the original code to permit various other pd ee controller options with the new solver. LM solver is now the default.
Users can modify the IK sol...
github.com
August 20, 2025 at 1:08 AM
Big thanks to Jeremy Morgan (PhD @ USC) for helping massively accelerate the IK solvers in ManiSkill! Open source really helps grow this project (and i get to learn tons of new things like IK optimization)
github.com/haosulab/Man...
github.com/haosulab/Man...
might be the first time I made it personally on the trending developers list on github! One of the contributing factors being the lerobot sim2real code
July 19, 2025 at 3:12 AM
might be the first time I made it personally on the trending developers list on github! One of the contributing factors being the lerobot sim2real code
Reposted by Stone Tao
Combining sim to real with cheap hardware like the so-100 really is making cutting edge robot learning so accessible and its wonderful
Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
July 5, 2025 at 3:20 AM
Combining sim to real with cheap hardware like the so-100 really is making cutting edge robot learning so accessible and its wonderful
Reposted by Stone Tao
My latest post: The American DeepSeek Project
Build fully open models in the US in the next two years to enable a flourishing, global scientific AI ecosystem to balance China's surge in open-source and an alternative to building products ontop of leading closed models.
buff.ly/kvJQE3I
Build fully open models in the US in the next two years to enable a flourishing, global scientific AI ecosystem to balance China's surge in open-source and an alternative to building products ontop of leading closed models.
buff.ly/kvJQE3I
July 4, 2025 at 2:06 PM
My latest post: The American DeepSeek Project
Build fully open models in the US in the next two years to enable a flourishing, global scientific AI ecosystem to balance China's surge in open-source and an alternative to building products ontop of leading closed models.
buff.ly/kvJQE3I
Build fully open models in the US in the next two years to enable a flourishing, global scientific AI ecosystem to balance China's surge in open-source and an alternative to building products ontop of leading closed models.
buff.ly/kvJQE3I
Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
July 4, 2025 at 6:31 PM
Awesome to see people reproducing the accessible LeRobot zero-shot sim2real project! Trained for just 90 min in ManiSkill and deployed directly in real. Sim2real is not easy, but very rewarding when it works
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Original post by Jianwei Zhang on LinkedIn www.linkedin.com/posts/jianwe...
Excited to announce that I will be interning at @nvidia research this summer on robotics/embodied AI! I’ll be in seattle for the summer, let me know if you want to meet up and chat! 🦾
June 28, 2025 at 5:37 PM
Excited to announce that I will be interning at @nvidia research this summer on robotics/embodied AI! I’ll be in seattle for the summer, let me know if you want to meet up and chat! 🦾
Rest of my lab has various presentations at RSS2025, please check out their awesome work!
June 25 EEB 248 @ 10am - Hardware Optimization for In-Hand Rotation by K. Fay
June 25 EEB 248 @ 10am - Hardware Optimization for In-Hand Rotation by K. Fay
June 22, 2025 at 6:51 PM
Rest of my lab has various presentations at RSS2025, please check out their awesome work!
June 25 EEB 248 @ 10am - Hardware Optimization for In-Hand Rotation by K. Fay
June 25 EEB 248 @ 10am - Hardware Optimization for In-Hand Rotation by K. Fay
The sim2real demo had some mixed success, hampered primarily by the lighting conditions of the outdoors.
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
June 22, 2025 at 4:29 PM
The sim2real demo had some mixed success, hampered primarily by the lighting conditions of the outdoors.
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
At least it worked sometimes! Hindsight says that despite the weather, low-cost nature, only 1 hour of training, anything working is a miracle
I’ll be at #RSS2025 from June 21 to June 23!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
June 20, 2025 at 7:44 PM
I’ll be at #RSS2025 from June 21 to June 23!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
I’ll be giving a presentation on ManiSkill3 on June 21, 5:30 PM
We will also have two live demo sessions, on June 21, 12:30-2:00PM and 6:30-8:00PM. Swing by to see live demos of zero shot RGB sim2real, cool sim demos, and VR teleop!
Reposted by Stone Tao
Our computer vision textbook is now available for free online here:
visionbook.mit.edu
We are working on adding some interactive components like search and (beta) integration with LLMs.
Hope this is useful and feel free to submit Github issues to help us improve the text!
visionbook.mit.edu
We are working on adding some interactive components like search and (beta) integration with LLMs.
Hope this is useful and feel free to submit Github issues to help us improve the text!
Foundations of Computer Vision
The print version was published by
visionbook.mit.edu
June 15, 2025 at 3:45 PM
Our computer vision textbook is now available for free online here:
visionbook.mit.edu
We are working on adding some interactive components like search and (beta) integration with LLMs.
Hope this is useful and feel free to submit Github issues to help us improve the text!
visionbook.mit.edu
We are working on adding some interactive components like search and (beta) integration with LLMs.
Hope this is useful and feel free to submit Github issues to help us improve the text!
Reposted by Stone Tao
it's happening.gif @stonet2000.bsky.social
June 15, 2025 at 2:58 AM
it's happening.gif @stonet2000.bsky.social
Reposted by Stone Tao
Rendering nerds! Check out our latest work "Vector-Valued Monte Carlo Integration Using Ratio Control Variates" that has just gotten the best paper award at SIGGRAPH 2025. This paper presents a method that reduces variance of a wide range of rendering and diff. rendering tasks with negligible cost.
June 14, 2025 at 5:26 PM
Rendering nerds! Check out our latest work "Vector-Valued Monte Carlo Integration Using Ratio Control Variates" that has just gotten the best paper award at SIGGRAPH 2025. This paper presents a method that reduces variance of a wide range of rendering and diff. rendering tasks with negligible cost.
Code/tutorial released for🤗LeRobot zero-shot visual sim2real Reinforcement Learning!
Train your SO100 robot in ManiSkill's fast simulator+renderer💨 and deploy a cube picking model zero-shot. Just in time for the LeRobot Hackathon, happy hacking!
github.com/StoneT2000/l...
Train your SO100 robot in ManiSkill's fast simulator+renderer💨 and deploy a cube picking model zero-shot. Just in time for the LeRobot Hackathon, happy hacking!
github.com/StoneT2000/l...
GitHub - StoneT2000/lerobot-sim2real: lerobot sim2real code
lerobot sim2real code. Contribute to StoneT2000/lerobot-sim2real development by creating an account on GitHub.
github.com
June 13, 2025 at 10:53 PM
Code/tutorial released for🤗LeRobot zero-shot visual sim2real Reinforcement Learning!
Train your SO100 robot in ManiSkill's fast simulator+renderer💨 and deploy a cube picking model zero-shot. Just in time for the LeRobot Hackathon, happy hacking!
github.com/StoneT2000/l...
Train your SO100 robot in ManiSkill's fast simulator+renderer💨 and deploy a cube picking model zero-shot. Just in time for the LeRobot Hackathon, happy hacking!
github.com/StoneT2000/l...
longer uncut video 17/18 successes
increased the spawn area size and tuned it to be less occluded. Works well!
increased the spawn area size and tuned it to be less occluded. Works well!
June 12, 2025 at 2:22 AM
longer uncut video 17/18 successes
increased the spawn area size and tuned it to be less occluded. Works well!
increased the spawn area size and tuned it to be less occluded. Works well!
sneak peek of what Xander and I will show next week at RSS 2025 for the maniskill demo session:
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
June 11, 2025 at 5:56 PM
sneak peek of what Xander and I will show next week at RSS 2025 for the maniskill demo session:
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Zero shot visual sim2real (basic) manipulation, one camera and <1 hour of RL in sim with SO100
~3 seconds to pick random color cubes, could be faster but goal is accessibility and low cost!
Is there any research on replay buffer sizes for off policy (sac) RL algorithms? How come we have just assumed that 1M replay buffer size is reasonable for the majority of tasks we test on (in robotics typically). Why not smaller? Can we make it smaller?
June 4, 2025 at 9:58 PM
Is there any research on replay buffer sizes for off policy (sac) RL algorithms? How come we have just assumed that 1M replay buffer size is reasonable for the majority of tasks we test on (in robotics typically). Why not smaller? Can we make it smaller?
Very good thread on how we got into this whole stem education / research cutting situation and its impacts by @davidbau.bsky.social
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
June 1, 2025 at 8:12 PM
Very good thread on how we got into this whole stem education / research cutting situation and its impacts by @davidbau.bsky.social
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Grants that are otherwise perfectly fine if you remove the “dei part” are being cut without being put back into stem education/research?!?!
x.com/davidbau/sta...
Nice to see the use of ManiSkill3 in this work!
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
May 17, 2025 at 12:23 AM
Nice to see the use of ManiSkill3 in this work!
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
Simulation is not just useful for RL training. It provides some good cheap deterministic test beds, perfect for testing imitation learning scaling laws at scale. Years of data in hours
Reposted by Stone Tao
For almost a decade, the environments used for RL research were slow, a huge research bottleneck in cost and throughput. The shift to fast envs will change:
- the speed of testing new algorithms and ideas
- decrease shoddy results caused by high eval costs
Everything is getting so exciting.
- the speed of testing new algorithms and ideas
- decrease shoddy results caused by high eval costs
Everything is getting so exciting.
May 16, 2025 at 6:28 PM
For almost a decade, the environments used for RL research were slow, a huge research bottleneck in cost and throughput. The shift to fast envs will change:
- the speed of testing new algorithms and ideas
- decrease shoddy results caused by high eval costs
Everything is getting so exciting.
- the speed of testing new algorithms and ideas
- decrease shoddy results caused by high eval costs
Everything is getting so exciting.
first thing I see on wechat channels when I get to China is a 1 hour uncut video of G1 speed walking on a track lmao
April 28, 2025 at 3:38 PM
first thing I see on wechat channels when I get to China is a 1 hour uncut video of G1 speed walking on a track lmao