ive personally never really given astrocytes much thought before so this was an interesting read

also fun to consider how advancements in AI are potentially leading to breakthroughs in reconceptualizing how the brain works (and hopefully the reverse happens as well)

their jupyter notebook shows how neuron-astrocyte network is equivalent to transformer (with random feature attention for approximating softmax attention)

github.com/kozleo/Build...

This work was made possible by our amazing coauthors (inc. @iscienceluvr.bsky.social @ptoncompmemlab.bsky.social) and Stability AI 🙏

This project was developed using an open lab approach where we publicly worked with volunteers in the MedARC Discord. We are continuing to work on neuroAI projects in-the-open, check out our lab website to learn more: medarc.ai/fmri

This work shows it is now practical for patients to undergo a single MRI scanning session and produce enough data to perform high-quality image reconstructions. This could enable novel clinical diagnosis and assessment approaches, including locked-in patient communication and BCIs.

If we use the full 40 hours instead of 1, we get SOTA performance for reconstruction and retrieval. We found that the 1-hour setting offered a good balance between scan duration and reconstruction performance, with notable improvements from first pre-training on other subjects.

Other innovations: 1. Mapping fMRI to OpenCLIP & reconstructing via new fine-tuned Stable Diffusion XL unCLIP model. 2. Merging previously independent high- and low-level pipelines into one. 3. Predict text captions for conditional guidance during a final refinement step.

How do you do shared-subject modeling when brains are differently shaped with different functional topography? We first do subject-specific ridge regression to a shared latent space, followed by subject-agnostic non-linear mapping, and train this single model end-to-end.

Past work trained independent models per person, with each person needing dozens of hours of training data in the MRI machine for high-quality results. We show it’s now possible to get high-quality reconstructions from a single visit to the MRI facility.

I am happy to say I'm not even leaving Princeton since I can work remotely for Stability; I'll become a visiting research scientist in the Norman lab to continue our existing projects while establishing new collaborations between MedARC and Princeton.

Also a massive thank you to my postdoc advisor Ken Norman for taking me into his lab, allowing me the freedom to pursue these crazy projects, and for being such a caring, hardworking, and insightful mentor throughout our time together.

I am extremely grateful for this opportunity. Huge thank you to Tanishq Abraham, PhD (CEO of MedARC) for his leadership in creating MedARC and Stability AI for being willing to invest in open neuroscience research.

This approach to doing research redefines the traditional research model, following similar footsteps as initiatives like EleutherAI, LAION, OpenBioML, and ML Collective.

arxiv link was pasted wrong 😓 heres correct link: arxiv.org/abs/2305.182...