Cem Anil
@cemanil.bsky.social
Research scientist at Anthropic.
PhD in machine learning from the University of Toronto and Vector Institute.
Prev: NVIDIA, Google
PhD in machine learning from the University of Toronto and Vector Institute.
Prev: NVIDIA, Google
Reposted by Cem Anil
Here's another oldie from 2020: an entire blog post about the weirdness of high-dimensional probability distributions.
With generative modelling having taken off as it has, understanding typicality and its implications is all the more important, but it still isn't talked about much in ML circles!
With generative modelling having taken off as it has, understanding typicality and its implications is all the more important, but it still isn't talked about much in ML circles!
Musings on typicality
A summary of my current thoughts on typicality, and its relevance to likelihood-based generative models.
sander.ai
November 22, 2024 at 12:25 AM
Here's another oldie from 2020: an entire blog post about the weirdness of high-dimensional probability distributions.
With generative modelling having taken off as it has, understanding typicality and its implications is all the more important, but it still isn't talked about much in ML circles!
With generative modelling having taken off as it has, understanding typicality and its implications is all the more important, but it still isn't talked about much in ML circles!
Reposted by Cem Anil
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
November 20, 2024 at 4:35 PM
How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this:
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️
Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢
🧵⬇️