This is a great example of research with immediate practical utility. Good work.

What are you baking up there?

Then whose take are you here for?

(2) unsanctioned distillation of the leading models (GPT, Claude, etc.) by the fast seconds namely DeepSeek, Qwen, Grok, and (3) the "vanillization" of LLM outputs driven by RLHF-derived human preference convergence. That said, this is a good empirical question in need of research!

Interesting observation. I would guess there are three factors at work (1) heavily overlapping pre-training datasets given how much of the scrapable internet is now consumed by LLMs, 1/2

For those looking for it, here's the link to the paper: arxiv.org/abs/2502.14739

Good share. I have been looking for follow up on Anthropic's work in the spring.

Awesome! Love to see these advances in encoder models - so much smaller and less compute-intense option for text classification tasks

This is a great question. At a minimum, could have three+ experts vote on which questions are incorrect and then omit those from the dataset going forward. This is true of several benchmarks -- what are the barriers to correction? Is it intentionally leaving canaries for target leakage?

While anecdotal, this is the type of side-by-side comparison I find very useful.