Mehdi Azabou
@mehdiazabou.bsky.social
Working on neuro-foundation models
Postdoc at Columbia | ML PhD, Georgia Tech | https://www.mehai.dev/
Postdoc at Columbia | ML PhD, Georgia Tech | https://www.mehai.dev/
Excited to announce the Foundation Models for the Brain and Body workshop at #NeurIPS2025! 🧠📈 🧪
We invite short papers or interactive demos on AI for neural, physiological or behavioral data.
Submit by Aug 22 👉 brainbodyfm-workshop.github.io
We invite short papers or interactive demos on AI for neural, physiological or behavioral data.
Submit by Aug 22 👉 brainbodyfm-workshop.github.io
July 11, 2025 at 5:01 PM
Excited to announce the Foundation Models for the Brain and Body workshop at #NeurIPS2025! 🧠📈 🧪
We invite short papers or interactive demos on AI for neural, physiological or behavioral data.
Submit by Aug 22 👉 brainbodyfm-workshop.github.io
We invite short papers or interactive demos on AI for neural, physiological or behavioral data.
Submit by Aug 22 👉 brainbodyfm-workshop.github.io
We find that the model’s latent representations carry meaningful information that reflect the anatomy and physiology of different regions and sub-types, even though it was never given any information about these distinctions!
April 25, 2025 at 10:14 PM
We find that the model’s latent representations carry meaningful information that reflect the anatomy and physiology of different regions and sub-types, even though it was never given any information about these distinctions!
We train POYO+ on the @alleninstitute.bsky.social brain observatory dataset. That's 256 mice, 6 visual brain areas and 13 genetically defined cellular sub-types.
This is x10 more data than POYO-1.
This is x10 more data than POYO-1.
April 25, 2025 at 10:14 PM
We train POYO+ on the @alleninstitute.bsky.social brain observatory dataset. That's 256 mice, 6 visual brain areas and 13 genetically defined cellular sub-types.
This is x10 more data than POYO-1.
This is x10 more data than POYO-1.
POYO+ adds support for regression, classification, and segmentation tasks. It can be trained on multiple tasks at the same time!
We query POYO+ when decoding, meaning that it can be queried to decode any number of tasks, and these tasks can be different depending on the context.
We query POYO+ when decoding, meaning that it can be queried to decode any number of tasks, and these tasks can be different depending on the context.
April 25, 2025 at 10:14 PM
POYO+ adds support for regression, classification, and segmentation tasks. It can be trained on multiple tasks at the same time!
We query POYO+ when decoding, meaning that it can be queried to decode any number of tasks, and these tasks can be different depending on the context.
We query POYO+ when decoding, meaning that it can be queried to decode any number of tasks, and these tasks can be different depending on the context.
POYO+ adds support for regular time series data through a value projection layer. We use it on calcium traces!
April 25, 2025 at 10:14 PM
POYO+ adds support for regular time series data through a value projection layer. We use it on calcium traces!
Thanks to everyone who came to Day 1 of the Workshop!
I had fun making this plot for the opening talk. It's exciting to see the exponential growth in the amount of pretraining data 🚀
I compiled a list of neuro-foundation models for EPhys and OPhys: github.com/mazabou/awes...
I had fun making this plot for the opening talk. It's exciting to see the exponential growth in the amount of pretraining data 🚀
I compiled a list of neuro-foundation models for EPhys and OPhys: github.com/mazabou/awes...
March 31, 2025 at 9:47 PM
Thanks to everyone who came to Day 1 of the Workshop!
I had fun making this plot for the opening talk. It's exciting to see the exponential growth in the amount of pretraining data 🚀
I compiled a list of neuro-foundation models for EPhys and OPhys: github.com/mazabou/awes...
I had fun making this plot for the opening talk. It's exciting to see the exponential growth in the amount of pretraining data 🚀
I compiled a list of neuro-foundation models for EPhys and OPhys: github.com/mazabou/awes...
We also tested our method on 🤖 simulated quadruped robots walking over procedurally generated terrains, and found that BAMS captures information about terrain type, morphology, policy, etc.
December 11, 2023 at 10:42 PM
We also tested our method on 🤖 simulated quadruped robots walking over procedurally generated terrains, and found that BAMS captures information about terrain type, morphology, policy, etc.
In mice for example, BAMS captures, in a fully unsupervised manner, information about short-term dynamics like chasing and huddling, and, long-term factors like mouse strain, and time of day. BAMS helps explain the behavioral state, and the many factors that modulate it.
December 11, 2023 at 10:41 PM
In mice for example, BAMS captures, in a fully unsupervised manner, information about short-term dynamics like chasing and huddling, and, long-term factors like mouse strain, and time of day. BAMS helps explain the behavioral state, and the many factors that modulate it.
BAMS ranks first overall on both the MABe 🐭 mouse triplet and 🪰 fruit flies benchmarks.
These multi-task benchmarks are designed to evaluate the learned representations of unsupervised methods using a set of 13 and 50 readout tasks for mice and flies resp.
These multi-task benchmarks are designed to evaluate the learned representations of unsupervised methods using a set of 13 and 50 readout tasks for mice and flies resp.
December 11, 2023 at 10:41 PM
BAMS ranks first overall on both the MABe 🐭 mouse triplet and 🪰 fruit flies benchmarks.
These multi-task benchmarks are designed to evaluate the learned representations of unsupervised methods using a set of 13 and 50 readout tasks for mice and flies resp.
These multi-task benchmarks are designed to evaluate the learned representations of unsupervised methods using a set of 13 and 50 readout tasks for mice and flies resp.
We introduce BAMS, a multiscale architecture that builds separate latent spaces to accommodate short- and long-term dynamics. BAMS is trained with 1. bootstrapping losses that encourage similarity of representations over time, and 2. HoA, a novel future action prediction loss.
December 11, 2023 at 10:39 PM
We introduce BAMS, a multiscale architecture that builds separate latent spaces to accommodate short- and long-term dynamics. BAMS is trained with 1. bootstrapping losses that encourage similarity of representations over time, and 2. HoA, a novel future action prediction loss.
Behavior is complex! It is shaped by various factors operating across different timescales. Immediate motivations can drive moment-to-moment activity, while long-term factors like time of day or some underlying complex objective, can influence behavior on broader scales.
December 11, 2023 at 10:39 PM
Behavior is complex! It is shaped by various factors operating across different timescales. Immediate motivations can drive moment-to-moment activity, while long-term factors like time of day or some underlying complex objective, can influence behavior on broader scales.
Automated methods for pose estimation and tracking have made it possible to process large amounts of video data. But, analyzing the extracted high-d, noisy trajectory data can be challenging, particularly in complex and naturalistic settings, where no annotations are available.
December 11, 2023 at 10:38 PM
Automated methods for pose estimation and tracking have made it possible to process large amounts of video data. But, analyzing the extracted high-d, noisy trajectory data can be challenging, particularly in complex and naturalistic settings, where no annotations are available.
How can we extract insights from behavior modulated by multiple complex factors? 🐭🪰🤖🏃
Check out our #NeurIPS2023 Spotlight Paper where we present a SSL method for learning multiscale representations of behavior! 🧠🟦
Link: multiscale-behavior.github.io
Check out our #NeurIPS2023 Spotlight Paper where we present a SSL method for learning multiscale representations of behavior! 🧠🟦
Link: multiscale-behavior.github.io
December 11, 2023 at 10:37 PM
How can we extract insights from behavior modulated by multiple complex factors? 🐭🪰🤖🏃
Check out our #NeurIPS2023 Spotlight Paper where we present a SSL method for learning multiscale representations of behavior! 🧠🟦
Link: multiscale-behavior.github.io
Check out our #NeurIPS2023 Spotlight Paper where we present a SSL method for learning multiscale representations of behavior! 🧠🟦
Link: multiscale-behavior.github.io
Finetuning our model is fast and efficient: We freeze 99% of the weights and only learn the unit embeddings for the units in a new recording. In many cases, we achieve SOTA performance through this simple approach.
October 26, 2023 at 1:17 PM
Finetuning our model is fast and efficient: We freeze 99% of the weights and only learn the unit embeddings for the units in a new recording. In many cases, we achieve SOTA performance through this simple approach.
But does having more data help when it's from diverse tasks and sources? Yes! In our analyses, we study the effect of scaling the model size and amount of data, and show that despite heterogeneity across animals and tasks, we obtain positive transfer across tasks and individuals.
October 26, 2023 at 1:16 PM
But does having more data help when it's from diverse tasks and sources? Yes! In our analyses, we study the effect of scaling the model size and amount of data, and show that despite heterogeneity across animals and tasks, we obtain positive transfer across tasks and individuals.
To compress our high-dimension input sequence of spike-level tokens, we use a PerceiverIO backbone to map a sequence of spikes to a sequence of behavior outputs.
October 26, 2023 at 1:15 PM
To compress our high-dimension input sequence of spike-level tokens, we use a PerceiverIO backbone to map a sequence of spikes to a sequence of behavior outputs.
We propose a new tokenization scheme where each individual spike is represented by a token that encodes the time of each spike event, and the corresponding unit embedding. This allows us to process the activity of a population of neurons without any binning!
October 26, 2023 at 1:13 PM
We propose a new tokenization scheme where each individual spike is represented by a token that encodes the time of each spike event, and the corresponding unit embedding. This allows us to process the activity of a population of neurons without any binning!
We learn a “unit embedding” space, where units can be aligned and organized across many sessions and different brains. Like with word embeddings, our model can learn the relationships between units within and across contexts.
October 26, 2023 at 1:10 PM
We learn a “unit embedding” space, where units can be aligned and organized across many sessions and different brains. Like with word embeddings, our model can learn the relationships between units within and across contexts.
Is a universal brain decoder possible? Can we train a decoding system that easily transfers to new individuals/tasks?
Check out our #NeurIPS2023 paper where we show that it’s possible to transfer from a large pretrained model to achieve SOTA! 🧠🟦
Link: poyo-brain.github.io 🧵
Check out our #NeurIPS2023 paper where we show that it’s possible to transfer from a large pretrained model to achieve SOTA! 🧠🟦
Link: poyo-brain.github.io 🧵
October 26, 2023 at 1:08 PM
Is a universal brain decoder possible? Can we train a decoding system that easily transfers to new individuals/tasks?
Check out our #NeurIPS2023 paper where we show that it’s possible to transfer from a large pretrained model to achieve SOTA! 🧠🟦
Link: poyo-brain.github.io 🧵
Check out our #NeurIPS2023 paper where we show that it’s possible to transfer from a large pretrained model to achieve SOTA! 🧠🟦
Link: poyo-brain.github.io 🧵