Francis Bach
@bachfrancis.bsky.social
Researcher in machine learning
Not all scaling laws are nice power laws. This month’s blog post: Zipf’s law in next-token prediction and why Adam (ok, sign descent) scales better to large vocab sizes than gradient descent: francisbach.com/scaling-laws...
September 27, 2025 at 2:57 PM
Not all scaling laws are nice power laws. This month’s blog post: Zipf’s law in next-token prediction and why Adam (ok, sign descent) scales better to large vocab sizes than gradient descent: francisbach.com/scaling-laws...
Jamais deux sans trois. Le département d’informatique de l’ENS toujours en forme. 131 km, 4500m de dénivelé, cette fois-ci avec du Beaufort aux ravitaillements!
Bravo Olivier Cappé! #LEtapeduTour
Bravo Olivier Cappé! #LEtapeduTour
July 21, 2025 at 6:56 AM
Jamais deux sans trois. Le département d’informatique de l’ENS toujours en forme. 131 km, 4500m de dénivelé, cette fois-ci avec du Beaufort aux ravitaillements!
Bravo Olivier Cappé! #LEtapeduTour
Bravo Olivier Cappé! #LEtapeduTour
Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/
francisbach.com/z-transform/
July 18, 2025 at 2:39 PM
Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/
francisbach.com/z-transform/
Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/
francisbach.com/z-transform/
July 18, 2025 at 2:24 PM
Tired of lengthy computations to derive scaling laws? This post is made for you: discover the sharpness of the z-transform!
francisbach.com/z-transform/
francisbach.com/z-transform/
What if AI isn’t about building solo geniuses, but designing social systems?
Michael Jordan advocates blending ML, economics, and uncertainty management to prioritize social welfare over mere prediction.
A must-read rethink.
arxiv.org/abs/2507.062...
Michael Jordan advocates blending ML, economics, and uncertainty management to prioritize social welfare over mere prediction.
A must-read rethink.
arxiv.org/abs/2507.062...
A Collectivist, Economic Perspective on AI
Information technology is in the midst of a revolution in which omnipresent data collection and machine learning are impacting the human world as never before. The word "intelligence" is being used as...
arxiv.org
July 13, 2025 at 1:10 PM
What if AI isn’t about building solo geniuses, but designing social systems?
Michael Jordan advocates blending ML, economics, and uncertainty management to prioritize social welfare over mere prediction.
A must-read rethink.
arxiv.org/abs/2507.062...
Michael Jordan advocates blending ML, economics, and uncertainty management to prioritize social welfare over mere prediction.
A must-read rethink.
arxiv.org/abs/2507.062...
Big thanks to the COLT 2025 organizers for an awesome event in Lyon! Here are the slides from my keynote this morning in case you’re curious about the references I mentioned: www.di.ens.fr/~fbach/fbach...
www.di.ens.fr
July 1, 2025 at 9:11 PM
Big thanks to the COLT 2025 organizers for an awesome event in Lyon! Here are the slides from my keynote this morning in case you’re curious about the references I mentioned: www.di.ens.fr/~fbach/fbach...
Reposted by Francis Bach
Register at PAISS 1-5 Sept 2025 @inria_grenoble with very talented speakers this year 🙂
paiss.inria.fr
cc @mvladimirova.bsky.social
paiss.inria.fr
cc @mvladimirova.bsky.social
June 9, 2025 at 8:55 PM
Register at PAISS 1-5 Sept 2025 @inria_grenoble with very talented speakers this year 🙂
paiss.inria.fr
cc @mvladimirova.bsky.social
paiss.inria.fr
cc @mvladimirova.bsky.social
Reposted by Francis Bach
The PAISS summer school is back with an incredible line of speakers (and more to come). Spread the word !
May 5, 2025 at 4:34 PM
The PAISS summer school is back with an incredible line of speakers (and more to come). Spread the word !
Reposted by Francis Bach
Épisode 5 de notre série "Les nouveaux visages de l’Académie des sciences" : La statistique, une science en son temps.
Retrouvez la vidéo sur Youtube : www.youtube.com/watch?v=2AhT...
Retrouvez la vidéo sur Youtube : www.youtube.com/watch?v=2AhT...
May 6, 2025 at 12:42 PM
Épisode 5 de notre série "Les nouveaux visages de l’Académie des sciences" : La statistique, une science en son temps.
Retrouvez la vidéo sur Youtube : www.youtube.com/watch?v=2AhT...
Retrouvez la vidéo sur Youtube : www.youtube.com/watch?v=2AhT...
Reposted by Francis Bach
Announcing : The 2nd International Summer School on Mathematical Aspects of Data Science
mathsdata2025.github.io
EPFL, Sept 1–5, 2025
Speakers:
Bach @bachfrancis.bsky.social
Bandeira
Mallat
Montanari
Peyré @gabrielpeyre.bsky.social
For PhD students & early-career researchers
Apply before May 15!
mathsdata2025.github.io
EPFL, Sept 1–5, 2025
Speakers:
Bach @bachfrancis.bsky.social
Bandeira
Mallat
Montanari
Peyré @gabrielpeyre.bsky.social
For PhD students & early-career researchers
Apply before May 15!
Mathematical Aspects of Data Science
Graduate Summer School - EPFL - Sept. 1-5, 2025
mathsdata2025.github.io
April 14, 2025 at 5:00 PM
Announcing : The 2nd International Summer School on Mathematical Aspects of Data Science
mathsdata2025.github.io
EPFL, Sept 1–5, 2025
Speakers:
Bach @bachfrancis.bsky.social
Bandeira
Mallat
Montanari
Peyré @gabrielpeyre.bsky.social
For PhD students & early-career researchers
Apply before May 15!
mathsdata2025.github.io
EPFL, Sept 1–5, 2025
Speakers:
Bach @bachfrancis.bsky.social
Bandeira
Mallat
Montanari
Peyré @gabrielpeyre.bsky.social
For PhD students & early-career researchers
Apply before May 15!
Reposted by Francis Bach
[NOUVELLE SERIE "LES NOUVEAUX VISAGES DE L'ACADEMIE DES SCIENCES]
Episode n°1 : Anne Canteaut : une architecte de la cryptographie moderne
www.youtube.com/watch?v=xyGC...
Episode n°1 : Anne Canteaut : une architecte de la cryptographie moderne
www.youtube.com/watch?v=xyGC...
April 1, 2025 at 10:01 AM
[NOUVELLE SERIE "LES NOUVEAUX VISAGES DE L'ACADEMIE DES SCIENCES]
Episode n°1 : Anne Canteaut : une architecte de la cryptographie moderne
www.youtube.com/watch?v=xyGC...
Episode n°1 : Anne Canteaut : une architecte de la cryptographie moderne
www.youtube.com/watch?v=xyGC...
Reposted by Francis Bach
Futur best seller!
March 28, 2025 at 8:08 AM
Futur best seller!
Characterizing finely the decay of eigenvalues of kernel matrices: many people need it, but explicit references are hard to find. This blog post reviews amazing asymptotic results from Harold Widom (1963!) and proposes new non-asymptotic bounds.
francisbach.com/spectrum-ker...
francisbach.com/spectrum-ker...
March 24, 2025 at 2:26 PM
Characterizing finely the decay of eigenvalues of kernel matrices: many people need it, but explicit references are hard to find. This blog post reviews amazing asymptotic results from Harold Widom (1963!) and proposes new non-asymptotic bounds.
francisbach.com/spectrum-ker...
francisbach.com/spectrum-ker...
The must-read introduction to PAC-Bayes!
I already advertised for this document when I posted it on arXiv, and later when it was published.
This week, with the agreement of the publisher, I uploaded the published version on arXiv.
Less typos, more references and additional sections including PAC-Bayes Bernstein.
arxiv.org/abs/2110.11216
This week, with the agreement of the publisher, I uploaded the published version on arXiv.
Less typos, more references and additional sections including PAC-Bayes Bernstein.
arxiv.org/abs/2110.11216
March 5, 2025 at 4:30 AM
The must-read introduction to PAC-Bayes!
Reposted by Francis Bach
🔬✨ Journée des Femmes et des filles de science✨🔬
À travers leurs parcours inspirants et leurs engagements, les académiciennes, et toutes les femmes et filles de science façonnent la recherche d’aujourd’hui et de demain.
Pour aller plus loin : urls.fr/PK5xg9
À travers leurs parcours inspirants et leurs engagements, les académiciennes, et toutes les femmes et filles de science façonnent la recherche d’aujourd’hui et de demain.
Pour aller plus loin : urls.fr/PK5xg9
urls.fr
February 11, 2025 at 9:05 AM
🔬✨ Journée des Femmes et des filles de science✨🔬
À travers leurs parcours inspirants et leurs engagements, les académiciennes, et toutes les femmes et filles de science façonnent la recherche d’aujourd’hui et de demain.
Pour aller plus loin : urls.fr/PK5xg9
À travers leurs parcours inspirants et leurs engagements, les académiciennes, et toutes les femmes et filles de science façonnent la recherche d’aujourd’hui et de demain.
Pour aller plus loin : urls.fr/PK5xg9
Reposted by Francis Bach
Check out our paper, with Lawrence Stewart and @bachfrancis.bsky.social
Link: arxiv.org/abs/2502.02996
1/8
Link: arxiv.org/abs/2502.02996
1/8
Building Bridges between Regression, Clustering, and Classification
Regression, the task of predicting a continuous scalar target y based on some features x is one of the most fundamental tasks in machine learning and statistics. It has been observed and...
arxiv.org
February 10, 2025 at 12:00 PM
Check out our paper, with Lawrence Stewart and @bachfrancis.bsky.social
Link: arxiv.org/abs/2502.02996
1/8
Link: arxiv.org/abs/2502.02996
1/8
Reposted by Francis Bach
If you're curious of what is "behind a term sheet", don't miss this account by the Cathay innovation team who led our recent series A at Bioptimus
medium.com/cathay-innov...
medium.com/cathay-innov...
Behind the Term Sheet: Bioptimus’ $41M Series A
The GPT of Biology Raising the Bar of AI-Driven Scientific Research
medium.com
February 9, 2025 at 10:54 AM
If you're curious of what is "behind a term sheet", don't miss this account by the Cathay innovation team who led our recent series A at Bioptimus
medium.com/cathay-innov...
medium.com/cathay-innov...
An inspirational talk by Michael Jordan: a refreshing, deep, and forward-looking vision for AI beyond LLMs.
www.youtube.com/live/W0QLq4q...
www.youtube.com/live/W0QLq4q...
February 7, 2025 at 7:56 AM
An inspirational talk by Michael Jordan: a refreshing, deep, and forward-looking vision for AI beyond LLMs.
www.youtube.com/live/W0QLq4q...
www.youtube.com/live/W0QLq4q...
Reposted by Francis Bach
Learning rate schedules seem mysterious? Why is the loss going down so fast during cooldown?
Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization.
A short thread on our latest paper 🚞
arxiv.org/abs/2501.18965
Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization.
A short thread on our latest paper 🚞
arxiv.org/abs/2501.18965
The Surprising Agreement Between Convex Optimization Theory and Learning-Rate Scheduling for Large Model Training
We show that learning-rate schedules for large model training behave surprisingly similar to a performance bound from non-smooth convex optimization theory. We provide a bound for the constant schedul...
arxiv.org
February 5, 2025 at 10:13 AM
Learning rate schedules seem mysterious? Why is the loss going down so fast during cooldown?
Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization.
A short thread on our latest paper 🚞
arxiv.org/abs/2501.18965
Turns out that this behaviour can be described with a bound from *convex, nonsmooth* optimization.
A short thread on our latest paper 🚞
arxiv.org/abs/2501.18965
Reposted by Francis Bach
Early stopping on validation loss? This leads to suboptimal calibration and refinement errors—but you can do better!
With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.
With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.
February 3, 2025 at 1:03 PM
Early stopping on validation loss? This leads to suboptimal calibration and refinement errors—but you can do better!
With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.
With @dholzmueller.bsky.social, Michael I. Jordan, and @bachfrancis.bsky.social, we propose a method that integrates with any model and boosts classification performance across tasks.
Reposted by Francis Bach
I've been eagerly awaiting this book for years! At last, a standalone and meticulous exposition of the current mathematical principles of machine learning, adorned with beautiful proofs. Well done and thank you,
@bachfrancis.bsky.social . Discover more here: francisbach.com/my-book-is-o...
@bachfrancis.bsky.social . Discover more here: francisbach.com/my-book-is-o...
My book is (at last) out! – Machine Learning Research Blog
francisbach.com
January 5, 2025 at 5:55 PM
I've been eagerly awaiting this book for years! At last, a standalone and meticulous exposition of the current mathematical principles of machine learning, adorned with beautiful proofs. Well done and thank you,
@bachfrancis.bsky.social . Discover more here: francisbach.com/my-book-is-o...
@bachfrancis.bsky.social . Discover more here: francisbach.com/my-book-is-o...
A happy author discovering the first hard copies
December 21, 2024 at 3:24 PM
A happy author discovering the first hard copies
Reposted by Francis Bach
My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
December 21, 2024 at 3:23 PM
My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
December 21, 2024 at 3:23 PM
My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
A blog post to celebrate and present it: francisbach.com/my-book-is-o...
Reposted by Francis Bach
Breaking news : L'Académie des sciences accueille 18 nouveaux membres dès 2025, avec une majorité féminine pour la 1ère fois depuis 1666 👩🔬 : un symbole fort pour la parité en science ! 💥
🔗 En savoir plus sur les nouveaux membres : urlr.me/mntDHX
🔗 En savoir plus sur les nouveaux membres : urlr.me/mntDHX
December 17, 2024 at 12:50 PM
Breaking news : L'Académie des sciences accueille 18 nouveaux membres dès 2025, avec une majorité féminine pour la 1ère fois depuis 1666 👩🔬 : un symbole fort pour la parité en science ! 💥
🔗 En savoir plus sur les nouveaux membres : urlr.me/mntDHX
🔗 En savoir plus sur les nouveaux membres : urlr.me/mntDHX