Why don’t neural networks learn all at once, but instead progress from simple to complex solutions? And what does “simple” even mean across different neural network architectures?

Sharing our new paper @iclr_conf led by Yedi Zhang with Peter Latham

arxiv.org/abs/2512.20607

Congrats to Ella for her new paper! She asked a really interesting question about how the brain represents uncertainty during hidden state inference, and in a lovely crossover with theoretical work, she shows that in mice, acetylcholine dynamics play a crucial role. www.biorxiv.org/content/10.1...

We watched mice follow scent trails drawn on an “endless” treadmill. By manipulating trail geometry/statistics, perturbing mouse nose & brain, and modeling behavior with a Bayesian framework, we show that mice use predictive (rather than reactive) strategies.

How does in-context learning emerge in attention models during gradient descent training?

Sharing our new Spotlight paper @icmlconf.bsky.social: Training Dynamics of In-Context Learning in Linear Attention
arxiv.org/abs/2501.16265

Led by Yedi Zhang with @aaditya6284.bsky.social and Peter Latham