Huge thanks to my amazing collaborators: Erin Kim, Sangho Suh, Ludwig Sidenmark, and Tovi Grossman. I'll present MaRginalia as my first CHI paper in the "Classroom Technology" session on Tuesday at 2:10PM-3:40PM in Room G302. If you are a note-taker, come check out our work and chat with us! 😉 (8/8)

MaRginalia introduces a new way to take notes and opens up further research for leveraging mixed reality for productivity and learning.
📄 Paper: arxiv.org/pdf/2501.16010
🎥 Video (4min): youtu.be/RBWh2WZriDY
🎙️ Talk (10min): youtu.be/EycV33cabZ0
👇 (7/8)

We conducted an exploratory user study with 12 participants to understand the system's usability and usefulness. Results found MaRginalia supports diverse workflows, has "good" usability on the System Usability Scale, and helps students stay engaged.👇 (6/8)

We introduced a Gaze+Pen interaction that operates similarly to a Wacom tablet. 👁️ Gaze is used as the initial cursor position. ✍️ Pen is used for adjusting the cursor, pushing buttons, and handwriting.👇 (5/8)

If the user misses something, they can quickly flip through prior slides and transcripts to catch up and see their annotations.👇 (4/8)

During the lecture, Marginalia automatically creates 📸 snapshots of slides and transcripts. When the user sees something important, they can directly annotate, and the content is seamlessly added to their notes.👇 (3/8)

Let us follow a user to see MaRginalia in action! The user can take handwritten notes on the tablet like they usually do, or take notes with mixed reality so that they can look at the lecture and stay engaged.👇 (2/8)

And to 4o's Ghibli style image recreations 👀

Need to convince my collaborators to join Bluesky ✊

I'm happy to chat more about the project here and at
@chi.acm.org #CHI2025!

📄 Preprint available on arXiv: arxiv.org/abs/2501.16010