If so, this would mean - as you also propose - that the responses indeed won't always be easily interpretable.

My current best guess is that cortex implements something like a JEPA and uses local error computations to implement credit assignment (or backprop if the system is hierarchical) the way Rafal Bogacz has suggested.

I also think the predictive processing model is wrong, but for slightly different reasons. These pertain to the way superficial and deep layers in cortex appear to interact that are not consistent with predictive processing (@loghyr.bsky.social‬ will have a preprint on this soon).

i.e. if the coding space of your population can represent bimodal distributions (in a non-trivial way), so can the prediction errors. And regarding the role of prediction errors in driving plasticity, the problem is the same for gradient descent in backprop.

Good point, but I don’t think that is different from other models of cortical function? Regarding the role of prediction errors in updating internal representations, the coding space of predictions, prediction errors, and internal representation just needs to match.

Intuitively, I think this would make sense – imagine opening a cookie jar to find a mouse having eaten your last cookie, I suspect your brain will prioritize the unexpected presence of the mouse, to only later be irked by the unexpected absence of the cookie.

My interpretation of this is that there likely is an inhibitory competition between negative and positive prediction errors that always ends up favoring positive prediction errors.

What I think Sonja’s paper very nicely shows is that presenting both positive and negative prediction errors (i.e. a stimulus substitution), drives only a positive prediction error response.

A negative prediction error neuron for stimulus surround (a la Rao&Ballard) might not respond to a visuomotor mismatch, etc.

There could be dedicated prediction error neurons to the different predictions (e.g. a visuomotor prediction, an audio-visual prediction, a stimulus surround prediction). It is still unclear how different predictions are combined.

Re 2) Also agreed. But there are multiple viable interpretations here. Assuming we let go of the idea of cortex as hierarchy (cortex is a hierarchy, like the earth is flat) – then there must are multiple predictive inputs.

The notion of ‘generalized predictive coding’ sounds similar to the assumption that if something is predictable in principle, the brain must predict it. I don’t think there ever was good evidence to support that assumption?

The two instances we have solid evidence for prediction errors are stimulus surround (i.e. statistics of natural images) and visuomotor coupling (i.e. physics of the world) – both likely behaviorally relevant.

Re 1) Correct, and why would it? We can easily construct (artificial) examples of perfectly predictable sequences of images we would intuitively agree the brain likely does not predict (think e.g. sequences of random visual noise that repeats every X s or so).

We had to learn (by trial and error) that even for low level audio-visual interactions, the mouse brain shows no signs of computing predictions or prediction errors unless the stimuli are also paired with a water reward or an air puff (pubmed.ncbi.nlm.nih.gov/34857950/).

And I suspect, if they were aversive, the brain might start predicting them – but they are probably just… boring? It is quite remarkable to see how quickly sensory responses to most stimuli ‘adapt’ away if the stimuli are presented repeatedly for days.