Rachel Bawden
@rachelbawden.bsky.social
Researcher in NLP in the ALMAnaCH team (Inria Paris)
Read Nathan's thread and (bsky.app/profile/nthn...) to get more details and the paper to get an even better picture: arxiv.org/abs/2510.25771.
Thrilled to release Gaperon, an open LLM suite for French, English and Coding 🧀
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
We trained 3 models - 1.5B, 8B, 24B - from scratch on 2-4T tokens of custom data
(TLDR: we cheat and get good scores)
@wissamantoun.bsky.social @rachelbawden.bsky.social @bensagot.bsky.social @zehavoc.bsky.social
November 12, 2025 at 11:18 PM
Read Nathan's thread and (bsky.app/profile/nthn...) to get more details and the paper to get an even better picture: arxiv.org/abs/2510.25771.
The experiments are really interesting, giving insights into the training of such models, the impact of pre-training data, and the huge problem of test set leakage in pretraining data, a problem that we show has an impact on some very popular LLMs!
November 12, 2025 at 11:18 PM
The experiments are really interesting, giving insights into the training of such models, the impact of pre-training data, and the huge problem of test set leakage in pretraining data, a problem that we show has an impact on some very popular LLMs!
Congratulations to @nthngdy.bsky.social, @wissamantoun.bsky.social and Rian Touchent (who worked under the supervision of @zehavoc.bsky.social, @bensagot.bsky.social, Éric de La Clergerie and me) on the training of these generative models for French, English and code.
November 12, 2025 at 11:18 PM
Congratulations to @nthngdy.bsky.social, @wissamantoun.bsky.social and Rian Touchent (who worked under the supervision of @zehavoc.bsky.social, @bensagot.bsky.social, Éric de La Clergerie and me) on the training of these generative models for French, English and code.