Joern Stoehler
jstoehler.bsky.social
Joern Stoehler
@jstoehler.bsky.social
Alignment is too hard, we should do governance instead.

Leave me anonymous feedback at https://admonymous.co/joernstoehler
Why do you want to shift the median voter position away from 14 or 1 year olds via a voting age? How do you justify that kind of authoritarian conservatism?
November 14, 2025 at 10:45 PM
Anyway, do you still think your OP is true enough to be worth spreading? I.e. that doomers in 2015 said that AIs would never be able to model human values?
July 1, 2025 at 10:23 PM
Oh yeah I was also unsure back in 2019. Neutrally surprised, I'd say. Disappointed that it's not visibly enough to be used for alignment, happy it may be useful for e.g. brain upload research with a mostly helpful and rarely evil AI assistant.
July 1, 2025 at 10:22 PM
Bottom line: I am confident that EY in 2015 expected that AIs will eventually model human values, since that's a capability, and one that is instrumentally useful for deception or other predictons of human actions.
July 1, 2025 at 8:44 PM
The LW Link also explicitly notes that it is talking about complexity under the constraint of excluding learning the values from the environment. It does not say (on a quick read) that AIs will certainly be capable enough some day to know human values. That is said by EY elsewhere (not sure where).
July 1, 2025 at 8:39 PM
I was only very briefly confused about whether future AIs would ever learn human values, and was only mildly surprised how capable current/2023 LLMs were at that task already, even before AGI/ASI.
July 1, 2025 at 8:34 PM
Too complex (aka long) to insert without error into an AI's goal-slot, if it had a clear and editable goal-slot, is a different problem from being too complex for an AI to learn. Our techniques for setting the AI's goal are worse than the LLM/future ASI's techniques for learning.
July 1, 2025 at 8:32 PM
Can you name a doomer in 2015 who said this? Can you name an alignment researcher in 2025 who said that this vector represents the concept of evil in the same sense as the 2015 claim?
July 1, 2025 at 9:41 AM
I think it's good, bc the tech giants don't create more net consumer surplus globally due to their saved taxes in the EU than the EU would create social goods globally due to the earned taxes. So it's good here to move money via taxation.

(Remark: extinction due to AI doesn't change this calculus)
May 30, 2025 at 10:32 AM
How much can intergalactic probes passing through your local area be intercepted? Can pre-singularity secrets be kept forever?
November 28, 2024 at 11:02 AM
Which decision theories, incl bargaining strategies, are used by aliens/post-humans? Do they merge? Or conflict? Do posthuman minds experience a high structure low entropy environment, or a weirder world where physical laws are screened by simulation?
November 28, 2024 at 10:56 AM