Glen Berseth
@glenberseth.bsky.social
Assistant Prof at @UMontreal @mila-quebec.bsky.social @MontrealRobots
. CIFAR AI Chair, RL_Conference chair. Creating generalist problem-solving agents for the real world. He/him/il.
. CIFAR AI Chair, RL_Conference chair. Creating generalist problem-solving agents for the real world. He/him/il.
Pinned
Glen Berseth
@glenberseth.bsky.social
· Oct 8
Creating Generalist robotics policies (GRPs) is tricky. In this video (and code) I share how to create a GRP from scratch from some basic transformer code. This is the first step in my plan to create a course on large models and scaling for RL and Robotics.
This not only seems illegal but is a breach of the social contract and trust in this research system. I can't see any good reason for breaking the confidentially of research grants and reference letters.
Canadian researchers should be aware the there is a motion before the Parliamentary Standing Committee on Science and Research to force Tricouncils to hand over disaggregated peer review data on all applications:
Applicant names, profiles, demographics
Reviewers names, profiles, comments, and scores
Applicant names, profiles, demographics
Reviewers names, profiles, comments, and scores
October 31, 2025 at 2:55 PM
This not only seems illegal but is a breach of the social contract and trust in this research system. I can't see any good reason for breaking the confidentially of research grants and reference letters.
@pcastr.bsky.social is adding good clarity to the challenges of deep RL. Awesome stuff.
🚨The Formalism-Implementation Gap in RL research🚨
Lots of progress in RL research over last 10 years, but too much performance-driven => overfitting to benchmarks (like the ALE).
1⃣ Let's advance science of RL
2⃣ Let's be explicit about how benchmarks map to formalism
1/X
Lots of progress in RL research over last 10 years, but too much performance-driven => overfitting to benchmarks (like the ALE).
1⃣ Let's advance science of RL
2⃣ Let's be explicit about how benchmarks map to formalism
1/X
October 28, 2025 at 6:15 PM
@pcastr.bsky.social is adding good clarity to the challenges of deep RL. Awesome stuff.
I have been seeing the subtleties in this race play out as destroying relationships. Families can fall apart because of the pressure to keep going in this race. Take care of yourselves everyone.
October 25, 2025 at 6:08 PM
I have been seeing the subtleties in this race play out as destroying relationships. Families can fall apart because of the pressure to keep going in this race. Take care of yourselves everyone.
Does anyone in this universe know which human being accepts join requests for the #mlnews ml-news Google group? groups.google.com/g/ml-news. It says "This group is moderated and maintained by IMLS (www.machinelearning.org)." but that link just goes to the ICML webpage...
Machine Learning News - Google Groups
groups.google.com
October 19, 2025 at 9:55 PM
Does anyone in this universe know which human being accepts join requests for the #mlnews ml-news Google group? groups.google.com/g/ml-news. It says "This group is moderated and maintained by IMLS (www.machinelearning.org)." but that link just goes to the ICML webpage...
Surprise/empowerment/etc may be the fundamental objectives living organisms optimize, however it is very difficult to optimize these objectives. I will be giving a talk at international worlshop on #activeinference on how foundational models can help improve these methods.
October 17, 2025 at 3:13 PM
Surprise/empowerment/etc may be the fundamental objectives living organisms optimize, however it is very difficult to optimize these objectives. I will be giving a talk at international worlshop on #activeinference on how foundational models can help improve these methods.
For those interested in joining my lab, submit your application via the Mila form. This year I am particularly interested in students with skills/interests in robotics, reinforcement learning and, foundational models which will push forward the abilities of real world agents.
October 15, 2025 at 1:02 PM
For those interested in joining my lab, submit your application via the Mila form. This year I am particularly interested in students with skills/interests in robotics, reinforcement learning and, foundational models which will push forward the abilities of real world agents.
I am at #COLM2025 today to talk about AI, LLMs and simulation in the social simulation workshop. Come find me, happy to chat about all things AI, embodiment, and simulation.
October 10, 2025 at 2:50 PM
I am at #COLM2025 today to talk about AI, LLMs and simulation in the social simulation workshop. Come find me, happy to chat about all things AI, embodiment, and simulation.
The same could be said for science.
The single biggest epistemic challenge in the internet era is remaining calibrated about what "normal" people think while the internet throws up an infinite wall of crazy. Thousands of people sharing an absurd opinion on the internet tells you very little!
October 2, 2025 at 2:21 PM
The same could be said for science.
GRPO is more like REINFORCE than PPO.
1) It does not train a critic (no need with small variance)
2) The SCORE FUNCTION (difficult to call this an advantage) is over a batch using the same initial prompt (similar to the vine sample method from TRPO)
1) It does not train a critic (no need with small variance)
2) The SCORE FUNCTION (difficult to call this an advantage) is over a batch using the same initial prompt (similar to the vine sample method from TRPO)
October 1, 2025 at 12:49 AM
GRPO is more like REINFORCE than PPO.
1) It does not train a critic (no need with small variance)
2) The SCORE FUNCTION (difficult to call this an advantage) is over a batch using the same initial prompt (similar to the vine sample method from TRPO)
1) It does not train a critic (no need with small variance)
2) The SCORE FUNCTION (difficult to call this an advantage) is over a batch using the same initial prompt (similar to the vine sample method from TRPO)
On my way to South Korea for a week packed with robotics at the conference on Robot Learning, Humanoids2025, and the global forum on mechanical engineering.
September 24, 2025 at 12:23 PM
On my way to South Korea for a week packed with robotics at the conference on Robot Learning, Humanoids2025, and the global forum on mechanical engineering.
One of the most common logical fallacies I see is "GPUs are cooking, therefore progress." I see people with 1/10th the compute get 10x more progress because... they have a more thorough plan. #Moretimethinkinglesstimeburning
September 22, 2025 at 1:14 AM
One of the most common logical fallacies I see is "GPUs are cooking, therefore progress." I see people with 1/10th the compute get 10x more progress because... they have a more thorough plan. #Moretimethinkinglesstimeburning
I suggest going out and talking to real people. They provide a much richer signal.
The single biggest epistemic challenge in the internet era is remaining calibrated about what "normal" people think while the internet throws up an infinite wall of crazy. Thousands of people sharing an absurd opinion on the internet tells you very little!
September 8, 2025 at 9:30 PM
I suggest going out and talking to real people. They provide a much richer signal.
VLAs offer an avenue for generalist robot policies; however, naively following the action predictions leads to brittle or unsafe behaviours. We introduce VLAPS, which integrates model-based search with pre-trained VLA policies to improve performance without additional training.
August 23, 2025 at 5:52 PM
VLAs offer an avenue for generalist robot policies; however, naively following the action predictions leads to brittle or unsafe behaviours. We introduce VLAPS, which integrates model-based search with pre-trained VLA policies to improve performance without additional training.
Efficiency may be the most important. If we can't make these tools economical, they will not last.
AI efficiency is important. The median Gemini Apps text prompt in May 2025 used 0.24 Wh of energy (<9 seconds of TV watching) & 0.26 mL (~5 drops) of water. Over 12 months, we reduced the energy footprint of a median text prompt 33x, while improving quality:
cloud.google.com/blog/product...
cloud.google.com/blog/product...
August 21, 2025 at 10:22 PM
Efficiency may be the most important. If we can't make these tools economical, they will not last.
My lab at @montrealrobotics.bsky.social was honoured to present our recent work to @mark-carney.bsky.social and Even Solomon explaining how AI enables new robotics that will drive innovation in Canada. It was a pleasure getting into the details with a quick dive into deterministic policy gradients!
August 20, 2025 at 10:59 PM
My lab at @montrealrobotics.bsky.social was honoured to present our recent work to @mark-carney.bsky.social and Even Solomon explaining how AI enables new robotics that will drive innovation in Canada. It was a pleasure getting into the details with a quick dive into deterministic policy gradients!
Another fantastic Montreal Robotics Summer School! Thanks to our sponsors, organizers, and @mila-quebec.bsky.social, we doubled in size this year. Congratulations again to all the students who make this school happen, and for your progress in machine learning and robotics.
August 17, 2025 at 2:23 PM
Another fantastic Montreal Robotics Summer School! Thanks to our sponsors, organizers, and @mila-quebec.bsky.social, we doubled in size this year. Congratulations again to all the students who make this school happen, and for your progress in machine learning and robotics.
@rl-conference.bsky.social will be Montréal next year @umontreal-en.bsky.social!
August 7, 2025 at 2:06 AM
@rl-conference.bsky.social will be Montréal next year @umontreal-en.bsky.social!
After the LLM news using RL many are wondering whether progress in exploration or exploitation is needed to improve deep RL algorithms. This work introduces a new practical sub-optimality measure to understand how good an RL algorithm is at exploiting its experience.
August 5, 2025 at 3:10 AM
After the LLM news using RL many are wondering whether progress in exploration or exploitation is needed to improve deep RL algorithms. This work introduces a new practical sub-optimality measure to understand how good an RL algorithm is at exploiting its experience.
Dessert at @rl-conference.bsky.social .
July 31, 2025 at 1:35 PM
Dessert at @rl-conference.bsky.social .
I have been cooking some code for training large generalist robotics policies that is almost ready for sharing! I will be presenting a tutorial on the code in a few weeks at an #IVADO LLM/VLM agents boot camp. Come checkout the most agentic system with full robotics control.
ivado.ca/en/events/bo...
ivado.ca/en/events/bo...
July 29, 2025 at 12:38 AM
I have been cooking some code for training large generalist robotics policies that is almost ready for sharing! I will be presenting a tutorial on the code in a few weeks at an #IVADO LLM/VLM agents boot camp. Come checkout the most agentic system with full robotics control.
ivado.ca/en/events/bo...
ivado.ca/en/events/bo...
What is the best free service for compute? I am making some LLM and VLM fine-tuning examples in Python notebooks to share, but is there a free option with enough memory to load and train a model?
July 15, 2025 at 5:13 PM
What is the best free service for compute? I am making some LLM and VLM fine-tuning examples in Python notebooks to share, but is there a free option with enough memory to load and train a model?
Who is interested in planning some socials at @rl-conference.bsky.social this year?
Propose some socials for RLC! Research topics, affinity groups, niche interests, whatever comes to mind!
rl-conference.cc/call_for_soc...
rl-conference.cc/call_for_soc...
RLC Call for Workshops
rl-conference.cc
June 25, 2025 at 6:23 PM
Who is interested in planning some socials at @rl-conference.bsky.social this year?
Being unable to scale #DeepRL to solve diverse, complex tasks with large distribution changes has been holding back the #RL community. In this work, we demonstrate that with the right architecture and optimization adjustments, agents can maintain plasticity for large networks.
🚨 Excited to share our new work: "Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning"! 📈
We propose gradient interventions that enable stable, scalable learning, unlocking significant performance gains across agents and environments!
Details below 👇
We propose gradient interventions that enable stable, scalable learning, unlocking significant performance gains across agents and environments!
Details below 👇
June 24, 2025 at 1:01 AM
How can we make behavioural cloning (BC) achieve better combinatorial generalization on out-of-distribution goals?
We propose BYOL-γ: an auxiliary self-predictive loss to improve generalization for goal-conditioned BC. 🧵1/6
We propose BYOL-γ: an auxiliary self-predictive loss to improve generalization for goal-conditioned BC. 🧵1/6
June 21, 2025 at 2:32 PM
How can we make behavioural cloning (BC) achieve better combinatorial generalization on out-of-distribution goals?
We propose BYOL-γ: an auxiliary self-predictive loss to improve generalization for goal-conditioned BC. 🧵1/6
We propose BYOL-γ: an auxiliary self-predictive loss to improve generalization for goal-conditioned BC. 🧵1/6
Great dialogue between Michael Littman and Kate Hartley to provide an overview of how RL, AGI and imitation learning have arrived where they are and the ingredients to make "AGI". @rldmdublin2025.bsky.social
June 13, 2025 at 4:11 PM
Great dialogue between Michael Littman and Kate Hartley to provide an overview of how RL, AGI and imitation learning have arrived where they are and the ingredients to make "AGI". @rldmdublin2025.bsky.social