That’s true, and I think that’s exactly why Claude does so well proofs. It‘s just I happen to know first hand that proofs are a particularly difficult *kind* of program

Hey :) Seems like a lot of people moved here from old Twitter and I’m still catching up

If a tool is not popular, it’s uncompelling to argue that everyone is just mistaken. At some point you should ask why the tool isn’t useful (at the current cost/benefit point)

I do think a lot of people are in denial though!

I don’t think literally everyone should drop what they’re doing. But my sense is PL research as a whole is significantly under-reacting to AI. So I suppose I think *some more* PL people should bet on AI (but maybe not you!)

Happy to mail you a couple. Email me, my address is on my website

I think you’ve put your finger on the exact worldview mismatch because 5-10 years seems like an insanely long time horizon to me

Why constrain the grammar - just pull more samples and keep the ones that pass :p

8 years on, the future is here! xkcd.com/1813/

If I understand right, the private test set is only used during evaluation of the model - not available to the team doing the training

Seems almost certain it’s deliberately trained on math reasoning. The way the o-series models seem to work is by long CoT, with reinforcement learning to impose correct reasoning. Not much public about how o3 works internally, but Chollet has some speculation: arcprize.org/blog/oai-o3-...

A big jump on coding skill as well: