➕ Bonus: Theory can explain the “Platonic Representation Hypothesis”—the striking observation that different models often learn the same representations.

arxiv.org/abs/2405.07987

With the right assumptions (hopefully not too 🧚‍♀️), you get a rigorous mathematical explanation for why this happens 🤓

So—does this mean theory is doomed, and AI engineering is just a random walk? Not at all!

💡 @rpatrik96.bsky.social @wielandbrendel.bsky.social Randall Balestriero did an amazing job clarifying where theory can help practice—and where practice should inspire theory.

🤝

Honestly, the real achievement? We managed to sneak pics of both our pets into the paper. 😝 🐾
Check it out and let us know what you think!
arxiv.org/abs/2503.01824

At the core:
1️⃣ Identifiability theory
2️⃣ Compressed sensing
3️⃣ Quantitative interpretability

Our goal is a unified model for LRH, superposition, sparse coding, and AutoInterp—backed by theory and practical insights. 🧠🔍
arxiv.org/abs/2503.01824