Andrew Saxe
@saxelab.bsky.social
Professor at the Gatsby Unit and Sainsbury Wellcome Centre, UCL, trying to figure out how we learn
To understand what is happening in the RNN, we extract automata from its hidden representations during training, which visualize the computational algorithm as it is being developed.
(5/11)
(5/11)
July 14, 2025 at 9:25 PM
To understand what is happening in the RNN, we extract automata from its hidden representations during training, which visualize the computational algorithm as it is being developed.
(5/11)
(5/11)
When training only on sequences up to length 10, we find complete generalization for any possible sequence length.
This cannot be explained by smooth interpolation of the training data, and suggests some kind of algorithm is being learned.
(4/11)
This cannot be explained by smooth interpolation of the training data, and suggests some kind of algorithm is being learned.
(4/11)
July 14, 2025 at 9:25 PM
When training only on sequences up to length 10, we find complete generalization for any possible sequence length.
This cannot be explained by smooth interpolation of the training data, and suggests some kind of algorithm is being learned.
(4/11)
This cannot be explained by smooth interpolation of the training data, and suggests some kind of algorithm is being learned.
(4/11)
Whoops, here's a working version of that starting video--dynamics visit a series of plateaus.
June 4, 2025 at 12:41 PM
Whoops, here's a working version of that starting video--dynamics visit a series of plateaus.