The rise of parallel computing hardware has made it increasingly important to understand which nonlinear state space models can be efficiently parallelized. Recent advances have shown that evaluating a state space model can be recast as solving a parallelizable optimization problem, and sometimes this approach yields dramatic speed-ups in evaluation time. However, the factors that govern the difficulty of these optimization problems remain unclear, limiting the larger adoption of the technique. In this work, we establish a precise relationship between the dynamics of a nonlinear system and the conditioning of its corresponding optimization formulation. We show that the predictability of a system, defined as the degree to which small perturbations in state influence future behavior, directly governs the number of optimization steps required for evaluation. In predictable systems, the state trajectory can be computed in $\mathcal{O}((\log T)^2)$ time, where $T$ is the sequence length, a major improvement over the conventional sequential approach. In contrast, chaotic or unpredictable systems exhibit poor conditioning, with the consequence that parallel evaluation converges too slowly to be useful. Importantly, our theoretical analysis demonstrates that for predictable systems, the optimization problem is always well-conditioned, whereas for unpredictable systems, the conditioning degrades exponentially as a function of the sequence length. We validate our claims through extensive experiments, providing practical guidance on when nonlinear dynamical systems can be efficiently parallelized, and highlighting predictability as a key design principle for parallelizable models.

Over three days from 15-17 October, the Champalimaud Foundation hosted the 2025 edition of the international Champalimaud Research Symposium (CRSy25), under the theme “Neuro-Cybernetics at Scale”. This symposium brought together leading researchers in neuroscience, robotics, machine learning, control theory, and theoretical neuroscience to explore how behaviour and intelligence emerge from complex feedback loops linking brains, bodies, and environments. Watch the wrap up video to get a glimpse of this year’s edition!

Self-related processes in the default mode network (DMN) have been viewed predominantly through a cognitive lens, often overlooking the embodied dimen…

Mateusz Kostecki

Cosyne 2025 Tutorial Session Sponsored by the Simons Foundation TOPIC: Foundations of Transformers in Neuroscience SPEAKER: Eva Dyer, Georgia Institute of Technology DATE: 27 March 2025 In this tutorial, we’ll introduce the fundamentals of transformers and their applications in neuroscience usin