Jay Alammar
@jayalammar.bsky.social
Writer http://jalammar.github.io. O'Reilly Author http://LLM-book.com. LLM Builder Cohere.com.
Pinned
Jay Alammar
@jayalammar.bsky.social
· Jan 27
The Illustrated DeepSeek-R1
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
Inside NeurIPS 2025: The Year’s AI Research, Mapped
New blog post!
NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters and
@cohere.com LLM-generated explanations that make it easier to grasp.
New blog post!
NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters and
@cohere.com LLM-generated explanations that make it easier to grasp.
November 3, 2025 at 4:39 PM
Inside NeurIPS 2025: The Year’s AI Research, Mapped
New blog post!
NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters and
@cohere.com LLM-generated explanations that make it easier to grasp.
New blog post!
NeurIPS 2025 papers are out—and it’s a lot to take in. This visualization lets you explore the entire research landscape interactively, with clusters and
@cohere.com LLM-generated explanations that make it easier to grasp.
Reposted by Jay Alammar
Excited to share that @jayalammar.bsky.social and I are writing the book “An Illustrated Guide to AI Agents” with @oreilly.bsky.social 🥳
Our new book will contain chapters on the fundamentals of agents (memory, tools, and planning), alongside more advanced concepts like RL and reasoning LLMs.
Our new book will contain chapters on the fundamentals of agents (memory, tools, and planning), alongside more advanced concepts like RL and reasoning LLMs.
October 13, 2025 at 3:11 PM
Excited to share that @jayalammar.bsky.social and I are writing the book “An Illustrated Guide to AI Agents” with @oreilly.bsky.social 🥳
Our new book will contain chapters on the fundamentals of agents (memory, tools, and planning), alongside more advanced concepts like RL and reasoning LLMs.
Our new book will contain chapters on the fundamentals of agents (memory, tools, and planning), alongside more advanced concepts like RL and reasoning LLMs.
The Illustrated Guide to AI Agents
New book announcement!
Thrilled that together with @maartengr.bsky.social , we're writing a new book titled “An Illustrated Guide to AI Agents” and published by @oreilly.bsky.social.
New book announcement!
Thrilled that together with @maartengr.bsky.social , we're writing a new book titled “An Illustrated Guide to AI Agents” and published by @oreilly.bsky.social.
October 13, 2025 at 3:13 PM
The Illustrated Guide to AI Agents
New book announcement!
Thrilled that together with @maartengr.bsky.social , we're writing a new book titled “An Illustrated Guide to AI Agents” and published by @oreilly.bsky.social.
New book announcement!
Thrilled that together with @maartengr.bsky.social , we're writing a new book titled “An Illustrated Guide to AI Agents” and published by @oreilly.bsky.social.
Reposted by Jay Alammar
The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we @icepfl.bsky.social @ethz.ch @cscsch.bsky.social ) built Apertus.
EPFL, ETH Zurich & CSCS just released Apertus, Switzerland’s first fully open-source large language model.
Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good.
Read more: actu.epfl.ch/news/apertus...
Trained on 15T tokens in 1,000+ languages, it’s built for transparency, responsibility & the public good.
Read more: actu.epfl.ch/news/apertus...
September 3, 2025 at 9:26 AM
The next generation of open LLMs should be inclusive, compliant, and multilingual by design. That’s why we @icepfl.bsky.social @ethz.ch @cscsch.bsky.social ) built Apertus.
The Illustrated GPT-OSS
New post! A visual tour of the architecture, message formatting, and reasoning of the latest GPT.
newsletter.languagemodels.co/p/the-illust...
New post! A visual tour of the architecture, message formatting, and reasoning of the latest GPT.
newsletter.languagemodels.co/p/the-illust...
August 19, 2025 at 1:20 PM
The Illustrated GPT-OSS
New post! A visual tour of the architecture, message formatting, and reasoning of the latest GPT.
newsletter.languagemodels.co/p/the-illust...
New post! A visual tour of the architecture, message formatting, and reasoning of the latest GPT.
newsletter.languagemodels.co/p/the-illust...
The legendary John Carmack at #upperbound:
- Current AI focus is RL (with Richard Sutton) solving Atari games
- Thinking in line with the Alberta Plan.
- It was a misstep to start working too low-level (e.g., at the cuda level). I kept stepping up the stack chain until now in pytorch
- Current AI focus is RL (with Richard Sutton) solving Atari games
- Thinking in line with the Alberta Plan.
- It was a misstep to start working too low-level (e.g., at the cuda level). I kept stepping up the stack chain until now in pytorch
May 23, 2025 at 4:41 PM
The legendary John Carmack at #upperbound:
- Current AI focus is RL (with Richard Sutton) solving Atari games
- Thinking in line with the Alberta Plan.
- It was a misstep to start working too low-level (e.g., at the cuda level). I kept stepping up the stack chain until now in pytorch
- Current AI focus is RL (with Richard Sutton) solving Atari games
- Thinking in line with the Alberta Plan.
- It was a misstep to start working too low-level (e.g., at the cuda level). I kept stepping up the stack chain until now in pytorch
Reposted by Jay Alammar
I'm really excited for this year's PyData London conference - there are some awesome talks on the schedule and I'm excited to hear the keynote speakers @jayalammar.bsky.social, Tony Wears, & Leanne Fitzpatrick
#pydata #datascience
#pydata #datascience
Unleash your inner data aficionado at PyData London 2025, 6-8 June at Convene Sancroft, St. Paul’s!
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
May 20, 2025 at 5:14 PM
I'm really excited for this year's PyData London conference - there are some awesome talks on the schedule and I'm excited to hear the keynote speakers @jayalammar.bsky.social, Tony Wears, & Leanne Fitzpatrick
#pydata #datascience
#pydata #datascience
Reposted by Jay Alammar
Unleash your inner data aficionado at PyData London 2025, 6-8 June at Convene Sancroft, St. Paul’s!
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
May 20, 2025 at 1:15 PM
Unleash your inner data aficionado at PyData London 2025, 6-8 June at Convene Sancroft, St. Paul’s!
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
We have 3 top flight keynotes lined up for you this year from @jayalammar.bsky.social, Leanne Kim Fitzpatrick and Tony Mears.
Just 17 days left. Book your tickets now!
pydata.org/london2025
Reposted by Jay Alammar
I'm excited to share the tech report for our @cohere.com @cohereforai.bsky.social Command A and Command R7B models. We highlight our novel approach to model training including self-refinement algorithms and model merging techniques at scale. Read more below! ⬇️
March 27, 2025 at 3:01 PM
I'm excited to share the tech report for our @cohere.com @cohereforai.bsky.social Command A and Command R7B models. We highlight our novel approach to model training including self-refinement algorithms and model merging techniques at scale. Read more below! ⬇️
Reposted by Jay Alammar
We've just released MMTEB, our multilingual upgrade to the MTEB Embedding Benchmark!
It's a huge collaboration between 56 universities, labs, and organizations, resulting in a massive benchmark of 1000+ languages, 500+ tasks, and a dozen+ domains.
Details in 🧵
It's a huge collaboration between 56 universities, labs, and organizations, resulting in a massive benchmark of 1000+ languages, 500+ tasks, and a dozen+ domains.
Details in 🧵
February 21, 2025 at 3:06 PM
We've just released MMTEB, our multilingual upgrade to the MTEB Embedding Benchmark!
It's a huge collaboration between 56 universities, labs, and organizations, resulting in a massive benchmark of 1000+ languages, 500+ tasks, and a dozen+ domains.
Details in 🧵
It's a huge collaboration between 56 universities, labs, and organizations, resulting in a massive benchmark of 1000+ languages, 500+ tasks, and a dozen+ domains.
Details in 🧵
Reposted by Jay Alammar
Did you know we continue to develop new content for the "Hands-On Large Language Models" book?
There's now even a free course available with
@deeplearningai.bsky.social!
There's now even a free course available with
@deeplearningai.bsky.social!
February 11, 2025 at 4:23 PM
Did you know we continue to develop new content for the "Hands-On Large Language Models" book?
There's now even a free course available with
@deeplearningai.bsky.social!
There's now even a free course available with
@deeplearningai.bsky.social!
Reposted by Jay Alammar
Do whales optimize their vocalizations for efficiency, just like human language? 🐋🎶 My latest study in
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...
Language-like efficiency in whale communication
Whale vocalizations follow efficiency rules seen in human language, revealing striking similarities in communication systems.
www.science.org
February 5, 2025 at 7:13 PM
Do whales optimize their vocalizations for efficiency, just like human language? 🐋🎶 My latest study in
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...
Science Advances (@science.org) suggests they do—following linguistic laws seen in human speech. 🧵 www.science.org/doi/10.1126/...
Reposted by Jay Alammar
We uncovered the same statistical structure that is a hallmark of human language in whale song, published today in Science. @inbalarnon.bsky.social @simonkirby.bsky.social @jennyallen13.bsky.social @clairenea.bsky.social @emma-carroll.bsky.social
www.science.org/doi/10.1126/...
www.science.org/doi/10.1126/...
February 6, 2025 at 7:15 PM
We uncovered the same statistical structure that is a hallmark of human language in whale song, published today in Science. @inbalarnon.bsky.social @simonkirby.bsky.social @jennyallen13.bsky.social @clairenea.bsky.social @emma-carroll.bsky.social
www.science.org/doi/10.1126/...
www.science.org/doi/10.1126/...
Reposted by Jay Alammar
One of my grand interpretability goals is to improve human scientific understanding by analyzing scientific discovery models, but this is the most convincing case yet that we CAN learn from model interpretation: Chess grandmasters learned new play concepts from AlphaZero's internal representations.
Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero
Artificial Intelligence (AI) systems have made remarkable progress, attaining super-human performance across various domains. This presents us with an opportunity to further human knowledge and improv...
arxiv.org
January 27, 2025 at 9:43 PM
One of my grand interpretability goals is to improve human scientific understanding by analyzing scientific discovery models, but this is the most convincing case yet that we CAN learn from model interpretation: Chess grandmasters learned new play concepts from AlphaZero's internal representations.
The Illustrated DeepSeek-R1
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
January 27, 2025 at 8:22 PM
The Illustrated DeepSeek-R1
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
Spent the weekend reading the paper and sorting through the intuitions. Here's a visual guide and the main intuitions to understand the model and the process that created it.
newsletter.languagemodels.co/p/the-illust...
Alphaxiv is an awesome way to discuss ML papers -- often with the authors themselves. Here's an intro and demo by @rajpalleti.bsky.social we shot at #Neurips2024
www.youtube.com/watch?v=-Kwl...
www.youtube.com/watch?v=-Kwl...
AlphaXiv - a great place to discuss ML papers
YouTube video by Jay Alammar
www.youtube.com
January 15, 2025 at 2:08 PM
Alphaxiv is an awesome way to discuss ML papers -- often with the authors themselves. Here's an intro and demo by @rajpalleti.bsky.social we shot at #Neurips2024
www.youtube.com/watch?v=-Kwl...
www.youtube.com/watch?v=-Kwl...
Reposted by Jay Alammar
The newest extremely strong embedding model based on ModernBERT-base is out: `cde-small-v2`. Both faster and stronger than its predecessor, this one tops the MTEB leaderboard for its tiny size!
Details in 🧵
Details in 🧵
January 14, 2025 at 1:21 PM
The newest extremely strong embedding model based on ModernBERT-base is out: `cde-small-v2`. Both faster and stronger than its predecessor, this one tops the MTEB leaderboard for its tiny size!
Details in 🧵
Details in 🧵
Floored that the repo for Hands-On Large Language Models is now at 3.6k Github stars!
And excited that professors are starting to use the book to teach LLM courses. Reach out to us if we can be of assistance!
And if you've liked the book, leave us a review on Amazon or Goodreads!
And excited that professors are starting to use the book to teach LLM courses. Reach out to us if we can be of assistance!
And if you've liked the book, leave us a review on Amazon or Goodreads!
January 14, 2025 at 4:46 PM
Floored that the repo for Hands-On Large Language Models is now at 3.6k Github stars!
And excited that professors are starting to use the book to teach LLM courses. Reach out to us if we can be of assistance!
And if you've liked the book, leave us a review on Amazon or Goodreads!
And excited that professors are starting to use the book to teach LLM courses. Reach out to us if we can be of assistance!
And if you've liked the book, leave us a review on Amazon or Goodreads!
SWE-Bench has been one of the most important tasks measuring the progress of agents tackling software engineering in 2024. I caught up with two of its creators, @ofirpress.bsky.social and Carlos E. Jimenez to share their ideas on the state of LLM-backed agents.
www.youtube.com/watch?v=bivZ...
www.youtube.com/watch?v=bivZ...
SWE-Bench authors reflect on the state of LLM agents at Neurips 2024
YouTube video by Jay Alammar
www.youtube.com
January 13, 2025 at 2:29 PM
SWE-Bench has been one of the most important tasks measuring the progress of agents tackling software engineering in 2024. I caught up with two of its creators, @ofirpress.bsky.social and Carlos E. Jimenez to share their ideas on the state of LLM-backed agents.
www.youtube.com/watch?v=bivZ...
www.youtube.com/watch?v=bivZ...
Reposted by Jay Alammar
OpenAI's o3: The grand finale of AI in 2024
A step change as influential as the release of GPT-4. Reasoning language models are the current and next big thing.
I explain:
* The ARC prize
* o3 model size / cost
* Dispelling training myths
* Extreme benchmark progress
A step change as influential as the release of GPT-4. Reasoning language models are the current and next big thing.
I explain:
* The ARC prize
* o3 model size / cost
* Dispelling training myths
* Extreme benchmark progress
o3: The grand finale of AI in 2024
A step change as influential as the release of GPT-4. Reasoning language models are the current big thing.
buff.ly
December 20, 2024 at 11:34 PM
OpenAI's o3: The grand finale of AI in 2024
A step change as influential as the release of GPT-4. Reasoning language models are the current and next big thing.
I explain:
* The ARC prize
* o3 model size / cost
* Dispelling training myths
* Extreme benchmark progress
A step change as influential as the release of GPT-4. Reasoning language models are the current and next big thing.
I explain:
* The ARC prize
* o3 model size / cost
* Dispelling training myths
* Extreme benchmark progress
Good morning #NeurIPS2024! Stop by the @cohere.com booth at 3PM today (Thursday) for a signed copy of Hands-On Large Language Models - it will introduce you to LLMs, their applications, as well as Cohere's Embed, Rerank, and Command-R models.
Come early as quantities are limited!
Come early as quantities are limited!
December 12, 2024 at 6:39 PM
Good morning #NeurIPS2024! Stop by the @cohere.com booth at 3PM today (Thursday) for a signed copy of Hands-On Large Language Models - it will introduce you to LLMs, their applications, as well as Cohere's Embed, Rerank, and Command-R models.
Come early as quantities are limited!
Come early as quantities are limited!
I'll be in the Cohere #NeurIPS2024 booth most of this afternoon. Come say hi, ask questions, and yes, we're hiring!
Tomorrow I'll be signing copies of my book at 3PM! Limited copies available!
Tomorrow I'll be signing copies of my book at 3PM! Limited copies available!
December 11, 2024 at 8:26 PM
I'll be in the Cohere #NeurIPS2024 booth most of this afternoon. Come say hi, ask questions, and yes, we're hiring!
Tomorrow I'll be signing copies of my book at 3PM! Limited copies available!
Tomorrow I'll be signing copies of my book at 3PM! Limited copies available!
Hi NeurIPS!
Explore ~4,500 NeurIPS papers in this interactive visualization:
jalammar.github.io/assets/neuri...
(Click on a point to see the paper on the website)
Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.
Explore ~4,500 NeurIPS papers in this interactive visualization:
jalammar.github.io/assets/neuri...
(Click on a point to see the paper on the website)
Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.
December 10, 2024 at 10:49 PM
Hi NeurIPS!
Explore ~4,500 NeurIPS papers in this interactive visualization:
jalammar.github.io/assets/neuri...
(Click on a point to see the paper on the website)
Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.
Explore ~4,500 NeurIPS papers in this interactive visualization:
jalammar.github.io/assets/neuri...
(Click on a point to see the paper on the website)
Uses @cohere.com models and @lelandmcinnes.bsky.social's datamapplot/umap to help make sense of the overwhelming scale of NeurIPS.
Sure to be thought provoking. The previous interview had fascinating thoughts on scifi (Dune Vs. Foundation), on AI competition for AI safety, and on successful scifi as a self-preventing prophecy.
Tim Ventura interviewed me about big perspectives on AI. Can't put the AI genie back in the bottle, so how to make it safe? Come explore ethical, legal & safety implications of artificial intelligence.
youtu.be/34_Tub6vXDk
youtu.be/34_Tub6vXDk
David Brin - Artificial Intelligence Safety
YouTube video by Tim Ventura
youtu.be
November 29, 2024 at 2:48 AM
Sure to be thought provoking. The previous interview had fascinating thoughts on scifi (Dune Vs. Foundation), on AI competition for AI safety, and on successful scifi as a self-preventing prophecy.
Excited to see you all at NeurIPS this year! Let's hang!
November 28, 2024 at 1:49 PM
Excited to see you all at NeurIPS this year! Let's hang!