Lessons in laughter:

Dozens of you have shared this lovely video with me over the past couple days. And I just love that it illustrates two baby lessons in one. 🧵

StarVector

StarVector is a foundation model for generating Scalable Vector Graphics (SVG) code from images and text. It utilizes a Vision-Language Modeling architecture to understand both visual and textual inputs, enabling high-quality vectorization and text-guided SVG creation.

Evidence that a well prompted LLM can help learning from a (small, single-subject) randomized controlled trial at Harvard: “here we show that students learn more than twice as much in less time with an AI tutor compared to an active learning classroom, while also being more engaged and motivated.”

Cool to see results extended! Last year, we identified a breakthrough in masked LM training where specialized syntactic heads emerge. We didn't think the same moment would be visible in decoder-only models, but Aoyama & @wegotlieb.bsky.social have found one, also marking a decline in "humanlikeness"

Sounds a bit like a fusion of System 2 thinking with a kind of working / short term memory?

Extrapolating @soniakmurthy.bsky.social findings: Individual differences in LLMs are low, which limits their diversity of thought. To me this suggests LLMs also have lower diversity of problem solving, creativity, etc. How to fix?

Good to see evidence validating OR rejecting the potential benefits of LLMs. Fair to say we should expect LLMs to enhance reasoning in many other jobs and skills.

one of the most intriguing projects i've been involved in: automated scientific discovery in an area (human/animal RL) I've been working on forever. can an LLM do the job of my grad students? if it is backed up by super smart scientists incl @pcastr.bsky.social @neurokim.bsky.social & kevin miller

Rings appear to be entangled.

A few implications of tricks like this:
1) We are still VERY early in the development of Reasoners
2) There is high value in understanding how humans solve problems & applying that to AI
3) Higher possibility of further exponential growth in AI capabilities as techniques for thinking traces compound

Making LLMs run efficiently can feel scary, but scaling isn’t magic, it’s math! We wanted to demystify the “systems view” of LLMs and wrote a little textbook called “How To Scale Your Model” which we’re releasing today. 1/n

Not so much a Python cheatsheet as mini-booklet. Covers everything from decorators to Panda to Pygame. Useful.

I went through my RL bookmarks, because it seems like finally the rest of the world has caught up to my world, I rediscovered this gem 💎 mpatacchiola.github.io/blog/2016/12... although I suspect nobody wants to learn RL this way now 😜

yugeten.github.io/posts/2025/0...

A vision researcher’s guide to some RL stuff: PPO & GRPO by Yuge (Jimmy) Shi

This is a deep dive into Proximal Policy Optimization (PPO), which is one of the most popular algorithm used in RLHF for LLMs, as well as Group Relative Policy Optimization (GRPO) proposed by the DeepSeek folks.

Fascinating - skimmed the paper and added to my deeper reading list. They've released data and implementation at github.com/Poirazi-Lab/...

Alternatives to backpropagation (how artifical neural networks learn) always catch my attention. Adding this to my reading list.

Beautiful

In case you missed it last week, we have a new paper about brain modularity: relating modular structures and modular functions. I'm so happy about these results that I've been reorienting my research programme to do more. Thread below with some extra speculations about where we might want to go next