The major barrier here is that the massive budgets get spent on AI researcher/engineer compensation and GPUs, leaving very little to pay for the best domain experts. I think this is a massive miscalculation. The first lab to realise this will quickly become the market leader.

It should therefore come as no surprise that new models like grok 3 and GPT 4.5 feel like small incremental improvements. The focus now should be on improving post-training data. Frankly, the quality kinda sucks even at the best labs.

The really interesting thing is that they’re not all made equal. Llama 8b distilled can solve medical cases that the Qwen-based R1 (and even R1 itself) cannot. World knowledge still matters for solving real problems and no open models beat Llama in that regard.

Say the benchmark has 100 questions, generate 64 responses per question and then pass@1 is total number correct / 6400.

5.7B tokens to solve 100 tasks??? I don’t understand why we’re thinking of this as being incredibly smart, when what this suggests is that it’s incredibly dumb.

Did you use oil?

“ILD, hyperglobulinemia & Lab abnormalities” sounds like SjD to me!

I just follow everyone and then spend all my time on the quiet posters feed (except when I want to come judge the yappers)

You better believe it. “California boomer” is the vibe I am getting from this guy.