Not all scaling laws are nice power laws. This month’s blog post: Zipf’s law in next-token prediction and why Adam (ok, sign descent) scales better to large vocab sizes than gradient descent: francisbach.com/scaling-laws...

Avec en prime une photo du lac de Roselend.

Jamais deux sans trois. Le département d’informatique de l’ENS toujours en forme. 131 km, 4500m de dénivelé, cette fois-ci avec du Beaufort aux ravitaillements!
Bravo Olivier Cappé! #LEtapeduTour

Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/

Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/

Characterizing finely the decay of eigenvalues of kernel matrices: many people need it, but explicit references are hard to find. This blog post reviews amazing asymptotic results from Harold Widom (1963!) and proposes new non-asymptotic bounds.
francisbach.com/spectrum-ker...

An inspirational talk by Michael Jordan: a refreshing, deep, and forward-looking vision for AI beyond LLMs.
www.youtube.com/live/W0QLq4q...

A happy author discovering the first hard copies

My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-o...