Can confirm this was a fun project! My favorite takeaway is that the (low-but-extensive) rank of a network can be used as a knob for controlling dimensionality while leaving single-neuron properties unchanged.
Now in PRX: Theory linking connectivity structure to collective activity in nonlinear RNNs! For neuro fans: conn. structure can be invisible in single neurons but shape pop. activity For low-rank RNN fans: a theory of rank=O(N) For physics fans: fluctuations around DMFT saddle⇒dimension of activity
Can confirm this was a fun project! My favorite takeaway is that the (low-but-extensive) rank of a network can be used as a knob for controlling dimensionality while leaving single-neuron properties unchanged.
Wanted to share a new version (much cleaner!) of a preprint on how connectivity structure shapes collective dynamics in nonlinear RNNs. Neural circuits have highly non-iid connectivity (e.g., rapidly decaying singular values, structured singular-vector overlaps), unlike classical random RNN models.
Wanted to share a new version (much cleaner!) of a preprint on how connectivity structure shapes collective dynamics in nonlinear RNNs. Neural circuits have highly non-iid connectivity (e.g., rapidly decaying singular values, structured singular-vector overlaps), unlike classical random RNN models.