My attempt at the next marginal unit of AI philosophy: LLMs are good at information but bad at insight. Barring a paradigm shift, humans retain a comparative advantage in insight.

🧵 rooted in @emollick.bsky.social's ideas about co-intellgience

davidmanuel.substack.com/p/ai-gives-...

‼️"o1-preview demonstrates superhuman performance in differential diagnosis, diagnostic clinical reasoning, and management reasoning, superior in multiple domains compared to prior model generations and human physicians."

And this is using vignettes, not multiple choice. arxiv.org/pdf/2412.10849

I am curious what others think about this, especially those using machine learning methods to contribute to nomothetic methods. Is my view of the limits of ML here outdated or based on a misunderstanding?

And I’ve been curious about something along the lines of what you asked more generally with Meehl and his metascience concepts.

Ie was he wrong in a substantive way or did it just fall out of fashion or fall to history, and there’s a there there?

There’s a fascinating later Meehl article where he put some qualifications on his own critiques of NHST, elaborating on strong vs weak NHST.

Maybe you mean, critiques of that framing as well. But just wanted to highlight it as when I bumped into this, it felt like hidden esoteric knowledge to me.

I am very interested in this side of metascience — about hashing out what is upstream of more or less valid inference about reality.

(Quoting this also because I am very excited about these papers! And although I am biased here I think they are definitely worth reading).