3.3 - Only the CRM component displays a significant association with behavior: The shared spectral pattern is increased when the animal is close to a decision point in the maze and is lowest in the home box

3.2 - With CRM, we maximize the correlation between power spectra while constraining correlations in the 60Hz band to zero. This reveals shared frequency-coupled representations between the regions that could not be captured with CCA (which latches on to shared noise in the recording)

2.2 - CRM can maximize correlations between runs of the same context while keeping correlations between different contexts in the schema at zero (b). We also maximize correlations of the same situation constraining within context correlations (c). This effectively factorizes neural representations

2.1 - BOLD patterns in mPFC represent schematic information (see Reagh et al. 2023): During movie-viewing the representational similarity between runs from the same schema (café/grocery videos) are highly correlated (panel a). Is specific information (e.g., of each café) also represented?

1.2 - To unambiguously show that pre-onset representations reflect upcoming words, we maximize the correlation between iEEG activity and word-embeddings while constraining the correlation with the previous word’s embedding (capturing the shared context) to zero. This cannot be achieved with CCA

Representations are central to cognitive neuroscience, but their signal can be subtle and overshadowed by confounding variables. CRM isolates them from multivariate series of observations (e.g., BOLD, embeddings, spectrograms) by maximizing correlations and keeping confounding variables uncorrelated

The strength of decreases in hippocampal power is correlated with the evidence for reinstatement of the upcoming state. We also observe enhanced conditional mutual information at and before event transitions: spectral patterns in hippocampus predict the spectral patterns in cortex

Computing the optimal alignment between event patterns and the memory scanning period (Dynamic Time Warping) we pinpoint exact moments of event transitions during memory scanning. Event transitions in memory are marked by power decreases in cortex and preceded by power decreases in hippocampus

During our naturalistic interview, we observe neural reinstatement of event patterns from the movie in the DMN when the corresponding scenes are described by the experimenter or the patient (left). During the memory scanning period, those patterns are unfurled in a forward direction (right)

We transform the neural data into slow components that are static over long periods of time (left) and segment them into events. Behavioral segmentation in a norming sample (button-presses at event boundaries) occurs ~1.3 sec after neural event boundaries are detected in the patient data (right)

We now investigate the neural mechanisms by which the continuous memory of a movie is unfurled as patients undergoing iEEG recording search for answers in memory. Patients first watch a movie; we then describe a scene in the movie and ask about a later scene, prompting them to scan their memory

Finally, we measure event boundary-ness continuously: The log probability of the “newline” token is sig. correlated with the log-probability of button-press in humans. Max. correlation of r = 0.372 with “Pieman” data (a,c; b,d=”Monkey in the Middle”; a,b=standard prompt; c,d="long" event prompt).

When we compare the model to the “consensus” solution derived from aggregated human responses, we find that the boundaries that were identified by GPT-3 are, on average, even closer to the consensus than boundaries identified by individual human annotators.

The model places new lines at event boundaries with significant overlap to human annotations. Panel a: Vertical lines are GPT-3-derived boundaries (5 replications in different colors) overlaid on average (top) and individual (bottom) human button presses. Figure b: "long" event prompt variation.