I have mentally tagged "Anthropic safety papers shoot themselves in just one or two toes, rather than the whole foot" as a reason to be more alarmed about the state of affairs

I do like the innoculation prompting tech tho, it just is neat

are Yud takes like this downstream of some world model of his that also leads to his beliefs about AI?

or like, are both his infants-aren't-sentient and AI-kills-all downstream of the "I am a bearer of elite gnosis" character trait, rather than world model

I would chose execution by adversarial poetry over the alternatives

this is kinda why I own TSMC and AMD, and not NVIDIA and also why I kinda hate myself

I mean if you're selling a surveillance state (TM) that probably sounds more attractive.

Both of these seem quite necessary?

Hrm I like this frame.

So as regards solutions to the latter (multi-agent learning, automated science laboratories, expensive AI-run ML experiments) we're trying to get a more human-like environment.

And as regards solutions to the latter (test-time training, ICL, etc) you're trying to make a more human-like brain.

this would have to be Omohundro 2008/2007, then Nick Bostrom 2014, then and also some Yud?

I'm not sure what the other canonical authors would be, there's probably someone I'm missing

oh man I kinda want to write this too, I have Strong Opinions on the old orthogonality thesis discussions

I think a blocker for me has been trying to figure out if it was right or wrong, but I think the right frame is more like -- from our perspective what is this just very confused about?

this bisk coming to you from more skeptical meditations on "superintelligence"