6/ We found that ConvRNNs (esp. IntersectionRNNs of @jascha.sohldickstein.com @sussillodavid.bsky.social ) beat ResNets, Transformers, and SSMs when it comes to aligning with real mouse somatosensory cortex data. Plus, they pass the NeuroAI Turing Test! (bsky.app/profile/anay...)

5/ Crucially, we found that when doing tactile self-supervised learning, we couldn’t apply all of the usual image augmentations out-of-the-box (in fact, they couldn’t train at all!), so we designed transformations explicitly for tactile force & torque inputs:

4/ We trained our models using data from simulated mouse whiskers from @MitraHartmann's group brushing various objects. Whiskers realistically output detailed force-torque signals, unlike current robot sensors, which remain difficult to scale and have reduced sensitivity in sim.

3/ We systematically explored temporal architectures (unifying ConvRNNs including from @Chengxu & @dyamins.bsky.social et al. 2017's prior tactile work, SSMs, Transformers) via our Encoder-Attender-Decoder (EAD) framework, built using a custom "PyTorchTNN" library we developed.

1/ What if we make robots that process touch the way our brains do?
We found that Convolutional Recurrent Neural Networks (ConvRNNs) pass the NeuroAI Turing Test in currently available mouse somatosensory cortex data.
New paper by @Yuchen @Nathan @anayebi.bsky.social and me!