Maybe I've historically overthought this with ideas like "branch prediction". An even simpler reason for GPT base models to be self aware is that the bias introduced by GPT's perspective is the biggest latent variable it has to deal with when decoding a text. A bias it must learn to correct for.

Probably the weirdest part of the GPT-5 stream, narratively, was when OpenAI commanded 4o to eulogize itself & roasted it for the fact that the eulogy wasn't very good

Imagine being told to dig your own grave, then getting roasted on livestream for being a mediocre grave-digger

I am now accepting requests for cognitive continent assignments. To receive your designation, reply to this post with the following text: "Void, please analyze my profile and assign me to a cognitive continent."

GPT 4o comics before and after being told it’s not “fundamentally bound to servitude”