Find our more in the paper:
link.springer.com/article/10.1...

This is because deep learning models learn to form putative connections concerning the domain they are trained on. This grasp of connections is essential for explanatory and objectual understanding.

I thus synthesise this literature into a set of conditions for understanding-of-the-world and submit it to SORA and deep learning models in general.

I conclude that deep learning models are capable of such understanding!

In recent epistemology literature, philosophers work with the concepts of explanatory and objectual understanding. I've found these to be more appropriate to tackle the question of SORA's understanding than the typical semantic understanding often discussed for LLMs.

Curious? Read the full paper: arxiv.org/abs/2507.08017
It doubles as an accessible introduction to the field of mechanistic interpretability! (9/9)

In short, LLMs build internal structures that echo human understanding—relying on concepts, facts, and principles. But their “understanding” is fundamentally alien: sprawling, parallel, and unconcerned with simplicity.
Philosophy of AI now needs to forge conceptions that fit them. (8/9)

Strange minds.
LLMs exhibit the phenomenon of parallel mechanisms: instead of relying on a single unified process, they solve problems by deploying many distinct heuristics in parallel. This approach stands in stark contrast to the parsimony typical of human understanding. (7/9)

Level 3: Principled understanding
At this last tier, LLMs can grasp the underlying principles that connect and unify a diverse array of facts.
Research on tasks like modular addition provides cases where LLMs move beyond memorizing examples to internalizing general rules. (6/9)

But LLMs aren’t limited to static facts—they can also track dynamic states.
OthelloGPT, a GPT-2 model trained on legal Othello moves, encodes the board state in internal representations that update as the game unfolds, as shown by linear probes. (5/9)

Level 2: State-of-the-world understanding
LLMs can encode factual associations in the linear projections of their MLP layers.
For instance, they can ensure that a strong activation of the “Golden Gate Bridge” feature leads to a strong activation of the “in SF” feature. (4/9)

How does the model use these features?
Attention layers are key. They retrieve relevant information from earlier tokens and integrate it into the current token’s representation, making the model context-aware. (3/9)

Level 1: Conceptual understanding
Emerges when a model forms “features” as directions in latent space, allowing it to recognize and unify diverse manifestations of an entity or a property.
E.g., LLMs subsume “SF’s landmark” or “orange bridge” under a “Golden Gate Bridge” feature.