I see what you mean now. We explored this question in simulations at some point. The general take-away was that noise did not alter the shape of the spectrum. It just reduced the range of dimensions that we could reliably detect.

Thanks! We use cross-validation and cross-subject analyses to address this. The effects we’re looking at generalize to held-out test data.

First, I think it’s an open question whether we should expect low-D representations for task purposes in general. Second, I think what Raj had in mind is that some tasks only require us to attend to a subset of features.

It’s still an open question whether you could explain these representations with a lower-dimensional nonlinear manifold. My hunch is there is no such simple manifold. But if anyone has suggestions for nonlinear methods to try, let us know! One challenge is that we need it to be cross-validated.

Yes, it’s generally thought that dimensionality governs a trade-off between robustness and expressivity. It’s possible that scale-free representations strike a balance between these two competing desiderata.

Agreed!

Our work demonstrates that fully understanding human brain representations requires a high-dimensional statistical approach—otherwise, we're just seeing the tip of the iceberg!

Why did so many previous studies report low dimensionality? 1. High-quality neural datasets are finally large enough to probe representations beyond just tens of dimensions! 2. Standard methods in cognitive neuroscience are insensitive to low-variance—but meaningful—dimensions.

We find this scale-free format throughout visual cortex, from V1 to V4 and beyond. We also find that the underlying dimensions are *shared* across individuals—suggesting that this high-dimensional structure reflects the fundamental format of the representational code.

Our new work, using the NSD dataset, shows that visual cortex representations are incredibly high-dimensional! Stimulus-related variance is distributed over PCs as a scale-free power law, with visual information detected over *thousands* of latent dimensions.

However, high-dimensional visual codes have strong theoretical benefits: they have higher expressive capacity and are flexible enough to support open-ended visual tasks. journals.plos.org/ploscompbiol...

Despite the millions of neurons in visual cortex, neuroscientists often report that representations are constrained to low-dimensional subspaces—suggesting that the goal of visual processing is to *compress* a torrent of high-dimensional sensory inputs into a more compact form.