Whoops, apparently, a transparent background does not work. Here is our model schematics

Finally, our model’s sequence-to-sequence behavior decoder allowed us to explore the temporal relationships between neural latent factors and behavior. The movement corrections generated latents that lag behind behavior by 90ms, suggesting a feedback-driven motor control strategy.

Comparing models on a force-field perturbation dataset, we found that model dynamics captures the movement plan (like LFADS), while behavior supervision captures transient corrections (like CEBRA). The best results are achieved with the combination of both.

We immediately achieved excellent results on Neural Latents Benchmark (neurallatents.github.io), outperforming both unsupervised as well as (semi-)supervised models on hand velocity reconstruction

We added behavior supervision in our Behavior Aligned Neural Dynamics (BAND) model to ensure that the neural code for movement correction is included in the latent variables, since movement corrections cause a substantial change in behavioral output.

We hypothesize that, unlike movement plans, corrections to a perturbation only transiently deviate from planned trajectories. As a result, these corrections would be reflected in relatively few spikes (i.e. account for a small neural variability).

In data from [Perich et al. 2018], we found that monkeys showed hand velocity oscillations during center-out reaches in a force-field. These oscillations were decodable from M1 (not PMd), but unsupervised models like LFADS struggled to capture them in the latent space.

Excited to share our new pre-print on bioRxiv, in which we reveal that feedback-driven motor corrections are encoded in small, previously missed neural signals.