Soufiane Hayou
@soufianehayou.bsky.social
Asst Professor at Johns Hopkins (AMS and DSAI). Previously: Simons Institute, Oxford stats, Polytechnique. I like to scale up things!
https://www.soufianehayou.com/
https://www.soufianehayou.com/
Reposted by Soufiane Hayou
AISTATS 2026 will be in Morocco!
July 30, 2025 at 8:07 AM
AISTATS 2026 will be in Morocco!
LoRA is amazing for finetuning large models cheaply, but WHERE you place the adapters makes a huge difference. Most people are just guessing where to put them (Attention, MLP, etc).
Meet "PLoP" (Precise LoRA Placement) 🎯, our new method for automatic LoRA placement 🧵
Meet "PLoP" (Precise LoRA Placement) 🎯, our new method for automatic LoRA placement 🧵
June 30, 2025 at 9:26 PM
LoRA is amazing for finetuning large models cheaply, but WHERE you place the adapters makes a huge difference. Most people are just guessing where to put them (Attention, MLP, etc).
Meet "PLoP" (Precise LoRA Placement) 🎯, our new method for automatic LoRA placement 🧵
Meet "PLoP" (Precise LoRA Placement) 🎯, our new method for automatic LoRA placement 🧵
The recent surge in available Research Scientist positions is correlated with the growing need for innovative approaches to scale and improve Large Language Models (LLMs). This trend is also driven by factors such as researchers leaving established companies for startups!
January 19, 2025 at 1:51 AM
The recent surge in available Research Scientist positions is correlated with the growing need for innovative approaches to scale and improve Large Language Models (LLMs). This trend is also driven by factors such as researchers leaving established companies for startups!
Tuyo (Narcos Theme) (Extended Version)
YouTube video by Rodrigo Amarante - Topic
www.youtube.com
January 17, 2025 at 1:00 AM
Are we hitting a wall with AI scaling? 🤔
That "plateau" you're seeing in scaling law charts might not be a fundamental limit, but a sign of suboptimal scaling strategies! I wrote a blogpost about this:
www.soufianehayou.com/blog/plateau...
That "plateau" you're seeing in scaling law charts might not be a fundamental limit, but a sign of suboptimal scaling strategies! I wrote a blogpost about this:
www.soufianehayou.com/blog/plateau...
January 10, 2025 at 11:40 PM
Are we hitting a wall with AI scaling? 🤔
That "plateau" you're seeing in scaling law charts might not be a fundamental limit, but a sign of suboptimal scaling strategies! I wrote a blogpost about this:
www.soufianehayou.com/blog/plateau...
That "plateau" you're seeing in scaling law charts might not be a fundamental limit, but a sign of suboptimal scaling strategies! I wrote a blogpost about this:
www.soufianehayou.com/blog/plateau...
Reposted by Soufiane Hayou
Speculative sampling accelerates inference in LLMs by drafting future tokens which are verified in parallel. With @vdebortoli.bsky.social , A. Galashov & @arthurgretton.bsky.social , we extend this approach to (continuous-space) diffusion models: arxiv.org/abs/2501.05370
January 10, 2025 at 4:30 PM
Speculative sampling accelerates inference in LLMs by drafting future tokens which are verified in parallel. With @vdebortoli.bsky.social , A. Galashov & @arthurgretton.bsky.social , we extend this approach to (continuous-space) diffusion models: arxiv.org/abs/2501.05370
People compare AI to past historic breakthroughs 🔄 (industrial revolution, internet, etc), but there's a crucial difference: In previous advancements, humans remained the most intelligent beings. This time, we're creating something that could surpass us 🤖. It's a singularity!⚡️
December 29, 2024 at 10:27 PM
People compare AI to past historic breakthroughs 🔄 (industrial revolution, internet, etc), but there's a crucial difference: In previous advancements, humans remained the most intelligent beings. This time, we're creating something that could surpass us 🤖. It's a singularity!⚡️