Cui Ding
@cuiding.bsky.social
Key Result 2: Information from Models
Using fine-tuned Qwen2.5-VL and TransOCR, we estimated the MI between images and word identity.
MI systematically drops: Full > Upper > Lower — perfectly mirroring human reading patterns! 🤯
Using fine-tuned Qwen2.5-VL and TransOCR, we estimated the MI between images and word identity.
MI systematically drops: Full > Upper > Lower — perfectly mirroring human reading patterns! 🤯
November 2, 2025 at 11:06 AM
Key Result 2: Information from Models
Using fine-tuned Qwen2.5-VL and TransOCR, we estimated the MI between images and word identity.
MI systematically drops: Full > Upper > Lower — perfectly mirroring human reading patterns! 🤯
Using fine-tuned Qwen2.5-VL and TransOCR, we estimated the MI between images and word identity.
MI systematically drops: Full > Upper > Lower — perfectly mirroring human reading patterns! 🤯
We model reading time as proportional to the number of visual “samples” needed to reduce uncertainty below a threshold ϕ.
Higher mutual information → fewer samples → faster reading.
Higher mutual information → fewer samples → faster reading.
November 2, 2025 at 11:06 AM
We model reading time as proportional to the number of visual “samples” needed to reduce uncertainty below a threshold ϕ.
Higher mutual information → fewer samples → faster reading.
Higher mutual information → fewer samples → faster reading.
📊 We quantify this using mutual information (MI) between visual input and word identity.
To test the theory, we created a reading experiment using the MoTR (Mouse-Tracking-for-Reading) paradigm 🖱️📖
We ran the study in both English and Chinese.
To test the theory, we created a reading experiment using the MoTR (Mouse-Tracking-for-Reading) paradigm 🖱️📖
We ran the study in both English and Chinese.
November 2, 2025 at 11:06 AM
📊 We quantify this using mutual information (MI) between visual input and word identity.
To test the theory, we created a reading experiment using the MoTR (Mouse-Tracking-for-Reading) paradigm 🖱️📖
We ran the study in both English and Chinese.
To test the theory, we created a reading experiment using the MoTR (Mouse-Tracking-for-Reading) paradigm 🖱️📖
We ran the study in both English and Chinese.
👀Ever wondered how visual information quality affects reading and language processing?
Our new #EMNLP2025 paper with @wegotlieb.bsky.social, Lena Jäger -- “Modeling Bottom-up Information Quality during Language Processing”, bridges psycholinguistics and multimodal LLMs.
🧠💡👇
arxiv.org/pdf/2509.17047
Our new #EMNLP2025 paper with @wegotlieb.bsky.social, Lena Jäger -- “Modeling Bottom-up Information Quality during Language Processing”, bridges psycholinguistics and multimodal LLMs.
🧠💡👇
arxiv.org/pdf/2509.17047
November 2, 2025 at 11:06 AM
👀Ever wondered how visual information quality affects reading and language processing?
Our new #EMNLP2025 paper with @wegotlieb.bsky.social, Lena Jäger -- “Modeling Bottom-up Information Quality during Language Processing”, bridges psycholinguistics and multimodal LLMs.
🧠💡👇
arxiv.org/pdf/2509.17047
Our new #EMNLP2025 paper with @wegotlieb.bsky.social, Lena Jäger -- “Modeling Bottom-up Information Quality during Language Processing”, bridges psycholinguistics and multimodal LLMs.
🧠💡👇
arxiv.org/pdf/2509.17047
Some of my colleagues are already very excited about this work!
June 4, 2025 at 5:58 PM
Some of my colleagues are already very excited about this work!
Participants must move their mouse over the text to reveal the words, while their cursor movements are recorded (similar to how eye movements are recorded in eye tracking). See below for an example MoTR trial.
March 7, 2025 at 10:26 PM
Participants must move their mouse over the text to reveal the words, while their cursor movements are recorded (similar to how eye movements are recorded in eye tracking). See below for an example MoTR trial.