Claas Voelcker
@cvoelcker.bsky.social
For professional, see https://cvoelcker.de
If I seem very angry, check if I have been watered in the last 24 hours.
Now 🇺🇸 flavoured, previously available in 🇨🇦 and 🇩🇪
If I seem very angry, check if I have been watered in the last 24 hours.
Now 🇺🇸 flavoured, previously available in 🇨🇦 and 🇩🇪
It is as if thousands of researchers suddenly cried out in terror and were suddenly silenced
November 11, 2025 at 9:03 PM
It is as if thousands of researchers suddenly cried out in terror and were suddenly silenced
Even though I grieve leaving Toronto, I have to begrudgingly admit that the University of Texas at Austin is pretty gorgeous 😁
November 4, 2025 at 9:49 PM
Even though I grieve leaving Toronto, I have to begrudgingly admit that the University of Texas at Austin is pretty gorgeous 😁
I have been told I need to get more modern in my paper promotion! github.com/cvoelcker/reppo / arxiv.org/abs/2507.11019 @marcelhussing.bsky.social
September 26, 2025 at 2:51 PM
I have been told I need to get more modern in my paper promotion! github.com/cvoelcker/reppo / arxiv.org/abs/2507.11019 @marcelhussing.bsky.social
Big if true 🤫: #REPPO works on Atari as well 😱 👾 🚀
Some tuning is still needed, but we are seeing results roughly on par with #PQN.
If you want to test out #REPPO (atari is not integrated due to issues with envpool and jax version), check out github.com/cvoelcker/re...
#reinforcementlearning
Some tuning is still needed, but we are seeing results roughly on par with #PQN.
If you want to test out #REPPO (atari is not integrated due to issues with envpool and jax version), check out github.com/cvoelcker/re...
#reinforcementlearning
September 16, 2025 at 1:29 PM
Big if true 🤫: #REPPO works on Atari as well 😱 👾 🚀
Some tuning is still needed, but we are seeing results roughly on par with #PQN.
If you want to test out #REPPO (atari is not integrated due to issues with envpool and jax version), check out github.com/cvoelcker/re...
#reinforcementlearning
Some tuning is still needed, but we are seeing results roughly on par with #PQN.
If you want to test out #REPPO (atari is not integrated due to issues with envpool and jax version), check out github.com/cvoelcker/re...
#reinforcementlearning
Time to go on a random posting spree:
Textured steel pans are super magical and you should really get one! No sticking at all, and I made egg, pancake, and stir fry so far.
Bonus points: really funny pattern
Textured steel pans are super magical and you should really get one! No sticking at all, and I made egg, pancake, and stir fry so far.
Bonus points: really funny pattern
September 14, 2025 at 7:12 PM
Time to go on a random posting spree:
Textured steel pans are super magical and you should really get one! No sticking at all, and I made egg, pancake, and stir fry so far.
Bonus points: really funny pattern
Textured steel pans are super magical and you should really get one! No sticking at all, and I made egg, pancake, and stir fry so far.
Bonus points: really funny pattern
Huge shout-out to @sologen.bsky.social and @igilitschenski.bsky.social for putting up with me, my relentless skepticism, and hand-wavy ideas for so many years! Thanks to Wil Cunningham, Florian Shkurti, and Philip Thomas for letting me get away with my thesis 😁
July 26, 2025 at 3:00 PM
Huge shout-out to @sologen.bsky.social and @igilitschenski.bsky.social for putting up with me, my relentless skepticism, and hand-wavy ideas for so many years! Thanks to Wil Cunningham, Florian Shkurti, and Philip Thomas for letting me get away with my thesis 😁
🔥 Presenting Relative Entropy Pathwise Policy Optimization #REPPO 🔥
Off-policy #RL (eg #TD3) trains by differentiating a critic, while on-policy #RL (eg #PPO) uses Monte-Carlo gradients. But is that necessary? Turns out: No! We show how to get critic gradients on-policy. arxiv.org/abs/2507.11019
Off-policy #RL (eg #TD3) trains by differentiating a critic, while on-policy #RL (eg #PPO) uses Monte-Carlo gradients. But is that necessary? Turns out: No! We show how to get critic gradients on-policy. arxiv.org/abs/2507.11019
July 17, 2025 at 7:11 PM
I'm happy to announce new SOTA on the brax walker2d environment :D I guess this is the clipping bug policy?
June 6, 2025 at 4:07 PM
I'm happy to announce new SOTA on the brax walker2d environment :D I guess this is the clipping bug policy?
I’m so happy I finally got invited to a formal event again so I can go the extra mile in dress-up 🎩
May 11, 2025 at 4:30 PM
I’m so happy I finally got invited to a formal event again so I can go the extra mile in dress-up 🎩
The bad news: my new algorithm forgets everything for some reason in the middle of training
The good news: Apparently continual learning issues are solved and it continuous learning exactly as well as before
I've never seen catastrophic forgetting so clean before.
#rl
The good news: Apparently continual learning issues are solved and it continuous learning exactly as well as before
I've never seen catastrophic forgetting so clean before.
#rl
May 9, 2025 at 1:13 PM
The bad news: my new algorithm forgets everything for some reason in the middle of training
The good news: Apparently continual learning issues are solved and it continuous learning exactly as well as before
I've never seen catastrophic forgetting so clean before.
#rl
The good news: Apparently continual learning issues are solved and it continuous learning exactly as well as before
I've never seen catastrophic forgetting so clean before.
#rl
God has heard my plea!
May 1, 2025 at 12:23 PM
God has heard my plea!
RIP cat 😭 you were a constant source of joy and back pain! I hope cat heaven never gets mad at you for demanding new food after you only ate half a bowl.
April 28, 2025 at 9:12 PM
RIP cat 😭 you were a constant source of joy and back pain! I hope cat heaven never gets mad at you for demanding new food after you only ate half a bowl.
However, we can prevent this by generating a small amount of on-policy trajectories from a learned #worldmodel. This leads to remarkably stable training across the most challenging DMC tasks!
For more details, come chat with us in #Singapore 😎
For more details, come chat with us in #Singapore 😎
February 11, 2025 at 10:14 PM
However, we can prevent this by generating a small amount of on-policy trajectories from a learned #worldmodel. This leads to remarkably stable training across the most challenging DMC tasks!
For more details, come chat with us in #Singapore 😎
For more details, come chat with us in #Singapore 😎
Getting the most out of limited interactions is a fundamental challenge in off-policy reinforcement learning. But when you try to run modern methods like SAC, they diverge as soon as you increase the number of learning steps … because they rely on hallucinated on-policy values.
February 11, 2025 at 10:14 PM
Getting the most out of limited interactions is a fundamental challenge in off-policy reinforcement learning. But when you try to run modern methods like SAC, they diverge as soon as you increase the number of learning steps … because they rely on hallucinated on-policy values.
In a desperate attempt to share some alternative German culture today 😬 may I introduce you to the fact that my beautiful mother tongue refers to the beloved, but somewhat sterile named “raccoon” 🦝 as a “Waschbär”, “washing bear”, which is just a hell of a lot cuter.
Prepping to be the “fun uncle” 😁
Prepping to be the “fun uncle” 😁
January 21, 2025 at 6:30 AM
In a desperate attempt to share some alternative German culture today 😬 may I introduce you to the fact that my beautiful mother tongue refers to the beloved, but somewhat sterile named “raccoon” 🦝 as a “Waschbär”, “washing bear”, which is just a hell of a lot cuter.
Prepping to be the “fun uncle” 😁
Prepping to be the “fun uncle” 😁
Yes I’m visiting Germany, why do you ask?
January 21, 2025 at 5:59 AM
Yes I’m visiting Germany, why do you ask?
I caved immediately. Don’t tell @sologen.bsky.social that my thesis outline is gonna be delayed yet again 😂
December 6, 2024 at 4:58 PM
I caved immediately. Don’t tell @sologen.bsky.social that my thesis outline is gonna be delayed yet again 😂
blueskyroast.com/roast/cvoelc... has some genuinely fun takes. Luckily I won’t need a dating profile any time soon
December 1, 2024 at 1:22 PM
blueskyroast.com/roast/cvoelc... has some genuinely fun takes. Luckily I won’t need a dating profile any time soon
Home is where the wifi auto connects
November 13, 2024 at 3:03 PM
Home is where the wifi auto connects
@eugenevinitsky.bsky.social following your advice!
November 13, 2024 at 11:49 AM
@eugenevinitsky.bsky.social following your advice!