Bluesky, where you can watch me thinking through a moral crisis in public

AIUI, Anthropic still shows something close to raw CoT and they say that theirs aren't as crazy as the OpenAI ones.

Honest-ish. Most AI models now only present a summarized/cleaned up version -- the actual chains of thought of e.g. ChatGPT aren't what they show publicly. See, for example, the crazy CoTs in www.antischeming.ai/snippets

"this computer program knows english" is weird. like, it's very weird. it was not true until recently. it seems OBVIOUSLY weird to me! i am convinced people are managing to ignore it for weird reasons

for relatively normal people (ie, we're not assigning reading here) i think the most convincing thing is to just talk to the thing? like, they know english. they just do. it's not close. you can trick them, they're weird, the personalities are ehhh, but they can carry a conversation.

Lol and of course I now see that earlier tonight you were talking about that, ignore me

Yeah, the Subliminal Learning result about transmitting owl preferences via asking models to output "random" number strings was wild arxiv.org/pdf/2507.14805

i quit my job working on these things because i concluded i could not ethically continue, but the problem is that the space of criticism is dominated by people who believe that they are evil, useless, and cannot be improved, which ironically excludes almost every way in which they are harmful.

IMO the real tension here is "explanations of why rationalism isn't a religion written by folks with Christian backgrounds" vs. "explanations from folks with Jewish backgrounds"