“OpenAI, Meta, Google etc. have billions of dollars, massive compute resources and world-class distribution. They don’t need to find a more efficient way to train models when their expensive solution is the only one. In fact, making it easier and cheaper to build LLMs would erode their advantages!”

With all these AI breakthroughs, I wonder what John Carmack is up to these days. My bets on him in the AGI race.