Neuro + ML PhD @ CMU'29 | BS in Math and in CS @ MIT' 23 & MEng' 24
Notably, our multimodal transformer independently rediscovered the brain’s modality organization, assigning higher visual weight to occipital regions, higher auditory weight to temporal areas, variable modality weight to switching areas, and balanced weight to jointly predicted areas.
Notably, our multimodal transformer independently rediscovered the brain’s modality organization, assigning higher visual weight to occipital regions, higher auditory weight to temporal areas, variable modality weight to switching areas, and balanced weight to jointly predicted areas.
The circular belts of regions that alternate their dominant sensory modality over time(shades of red below) , and the axis of regions along the occipito-temporal lobe that jointly track both modalities(shades of blue below), together form a bow-and-arrow-like pattern.
The circular belts of regions that alternate their dominant sensory modality over time(shades of red below) , and the axis of regions along the occipito-temporal lobe that jointly track both modalities(shades of blue below), together form a bow-and-arrow-like pattern.
DIP reveals a pair of “bows” of modality switching - one posterior bow encircling category-selective visual cortex and another anterior bow spanning dorso-lateral frontal areas (purple to pink in the plot are switching regions)
DIP reveals a pair of “bows” of modality switching - one posterior bow encircling category-selective visual cortex and another anterior bow spanning dorso-lateral frontal areas (purple to pink in the plot are switching regions)
Using large-scale fMRI data collected while participants watched movies, we developed two computational approaches relying on prediction performance to analyze temporal dynamics of sensory processing across cortical regions
Using large-scale fMRI data collected while participants watched movies, we developed two computational approaches relying on prediction performance to analyze temporal dynamics of sensory processing across cortical regions