Practically, our approach may enable us to adapt our semantic language decoders for people with impaired language comprehension. I’ve been working with
@mayalhenry.bsky.social and
@alexanderhuth.bsky.social
to test our approach in people with aphasia, stay tuned!

5/5

We tested our approach on neurologically healthy participants, and found that silent movies are nearly as effective as narrative stories for transferring semantic decoders. Scientifically, this adds to the growing evidence that semantic representations are shared between language and vision

4/5

Our new approach can decode language from a participant without language data! First we train decoders on reference participants with language data. Then we use *silent movies* to align brain responses across participants. Finally we decode new participant responses using the reference decoders

3/5

Language decoders that target semantic representations have the potential to help people with aphasia, who struggle to map concepts to lexical-phonological output. But many people with aphasia have impaired language comprehension, and current decoders are trained on brain responses to language

2/5