Do you expect 'value engineering' to need to happen at the foundational layers or to be easier via post-training methods?

It’s a big leap - every announcement over the last two years has stunned me. As the dust in my mind settles, these very smart models still can’t solve the core challenges when applying them, which is to ensure they have the right context to work with. Which ultimately is a user problem now.

Putting aside agentic approaches, the current transformer design doesn’t seem to allow an LLM to have an internal model to decide prioritization of new instructions. It has no concept of a pyramid of clarity. Which is also why it’s not good at poking holes in abstract concepts - is X incomplete?