Some example outputs:

This is "Qwen2.5 3B-base" model trained for 1000 RL steps only on CountDown task with correctness reward.

Checkpoint at huggingface.co/McGill-NLP/n...

Github Repo:
github.com/McGill-NLP/n...

YouTube Video:
www.youtube.com/playlist?lis...

and yes, we recreated DeepSeek R1-Zero style-training on CountDown in ~10h with one A100.

Introducing nanoAhaMoment: Karpathy-style, single file RL for LLM library (<700 lines)

- super hackable
- no TRL / Verl, no abstraction💆‍♂️
- Single GPU, full param tuning, 3B LLM
- Efficient (R1-zero countdown < 10h)

comes with a from-scratch, fully spelled out YT video [1/n]