Carl Allen
@carl-allen.bsky.social
Laplace Junior Chair, Machine Learning
ENS Paris. (prev ETH Zurich, Edinburgh, Oxford..)
Working on mathematical foundations/probabilistic interpretability of ML (what NNs learn🤷♂️, disentanglement🤔, king-man+woman=queen?👌…)
ENS Paris. (prev ETH Zurich, Edinburgh, Oxford..)
Working on mathematical foundations/probabilistic interpretability of ML (what NNs learn🤷♂️, disentanglement🤔, king-man+woman=queen?👌…)
Pinned
Carl Allen
@carl-allen.bsky.social
· Dec 18
Machine learning has made incredible breakthroughs, but our theoretical understanding lags behind.
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
Reposted by Carl Allen
How do tokens evolve as they are processed by a deep Transformer?
With José A. Carrillo, @gabrielpeyre.bsky.social and @pierreablin.bsky.social, we tackle this in our new preprint: A Unified Perspective on the Dynamics of Deep Transformers arxiv.org/abs/2501.18322
ML and PDE lovers, check it out!
With José A. Carrillo, @gabrielpeyre.bsky.social and @pierreablin.bsky.social, we tackle this in our new preprint: A Unified Perspective on the Dynamics of Deep Transformers arxiv.org/abs/2501.18322
ML and PDE lovers, check it out!
January 31, 2025 at 4:56 PM
How do tokens evolve as they are processed by a deep Transformer?
With José A. Carrillo, @gabrielpeyre.bsky.social and @pierreablin.bsky.social, we tackle this in our new preprint: A Unified Perspective on the Dynamics of Deep Transformers arxiv.org/abs/2501.18322
ML and PDE lovers, check it out!
With José A. Carrillo, @gabrielpeyre.bsky.social and @pierreablin.bsky.social, we tackle this in our new preprint: A Unified Perspective on the Dynamics of Deep Transformers arxiv.org/abs/2501.18322
ML and PDE lovers, check it out!
Machine learning has made incredible breakthroughs, but our theoretical understanding lags behind.
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
December 18, 2024 at 4:58 PM
Machine learning has made incredible breakthroughs, but our theoretical understanding lags behind.
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
We take a step towards unravelling its mystery by explaining why the phenomenon of disentanglement arises in generative latent variable models.
Blog post: carl-allen.github.io/theory/2024/...
If few-shot transfer is ur thing!
We maintain strong zero-shot transfer of CLIP / SigLIP across model size and data scale, while achieving up to 4x few-shot sample efficiency and up to +16% performance gains!
Fun project with @confusezius.bsky.social, @zeynepakata.bsky.social, @dimadamen.bsky.social and
@olivierhenaff.bsky.social.
Fun project with @confusezius.bsky.social, @zeynepakata.bsky.social, @dimadamen.bsky.social and
@olivierhenaff.bsky.social.
🤔 Can you turn your vision-language model from a great zero-shot model into a great-at-any-shot generalist?
Turns out you can, and here is how: arxiv.org/abs/2411.15099
Really excited to this work on multimodal pretraining for my first bluesky entry!
🧵 A short and hopefully informative thread:
Turns out you can, and here is how: arxiv.org/abs/2411.15099
Really excited to this work on multimodal pretraining for my first bluesky entry!
🧵 A short and hopefully informative thread:
November 28, 2024 at 5:07 PM
If few-shot transfer is ur thing!
Reposted by Carl Allen
November 23, 2024 at 11:08 AM
Reposted by Carl Allen
Just wondering (very neutrally!)… if authors have to review as a condition of submitting themselves, but they then review in *clear bad faith*, should their own submissions be rejected as if they had not reviewed? 🤔
#ICLR2025 @iclr-conf.bsky.social
#ICLR2025 @iclr-conf.bsky.social
November 21, 2024 at 1:40 PM
Just wondering (very neutrally!)… if authors have to review as a condition of submitting themselves, but they then review in *clear bad faith*, should their own submissions be rejected as if they had not reviewed? 🤔
#ICLR2025 @iclr-conf.bsky.social
#ICLR2025 @iclr-conf.bsky.social
Reposted by Carl Allen
The Country Land and Business Association says the new £1m cap on agricultural inheritance tax relief will "harm 70,000 farms". That's 1/3 of all farms.
What does the actual data show? Less than 500 farms/year will be pay more tax as a result of this change every year. Possibly as few as 100.
What does the actual data show? Less than 500 farms/year will be pay more tax as a result of this change every year. Possibly as few as 100.
October 31, 2024 at 7:05 PM
The Country Land and Business Association says the new £1m cap on agricultural inheritance tax relief will "harm 70,000 farms". That's 1/3 of all farms.
What does the actual data show? Less than 500 farms/year will be pay more tax as a result of this change every year. Possibly as few as 100.
What does the actual data show? Less than 500 farms/year will be pay more tax as a result of this change every year. Possibly as few as 100.