Evolve at the hyperscale!
Work co-led with Mattie Fellows and Juan Agustin Duque.
Made possible by #Isambard and AIRR

🌐 Website: eshyperscale.github.io
📝 Paper: alphaxiv.org/abs/2511.16652
💻 Code: github.com/ESHyperscale...
🥚NanoEgg : github.com/ESHyperscale... (train in int 😉)

Scaling LLM Reasoning with EGGROLL 🥚🧠📝

Using 🥚 to finetune RWKV-7 language models outperforms GRPO on Countdown and GSM8K ❗

🥚significantly outperformed GRPO on the Countdown task, achieving a 35% validation accuracy compared to GRPO's 23%❗

EGGROLL 🥚for RL 🎮🤖

🥚 is competitive with, and in many cases, better than OpenES performance, even before considering the vast speed-up!

🥚 matched OpenES on 7/16 environments and outperformed it on another 7/16

🥚's low-rank approach does not compromise ES performance

🥚EGGROLLing in the Deep with🚀 💯✕ Speedup

🥚 speed nearly reaches the throughput of pure batch inference, leaving OpenES far behind

🥚 reaches 91% of pure batch inference speed vs. OpenES reaching only 0.41%

The EGGROLL Recipe
🧠🛠️ We replace full-rank perturbations with low-rank ones. Each update is still high rank, maintaining expressivity with faster training

🥚 EGGROLL converges to the full-rank update at a fast rate of 1/rank. The method is effective even with a rank of 1

We use EGGROLL 🥚to train RNN language models from scratch using only integer datatypes (and no activation functions!), scaling population size from 64 to 262144

2 (🐔🐔) orders of magnitude larger than prior ES works❗