“If Anyone Builds it, Everyone Dies: Why Superhuman AI Would Kill us All.” A deeply troubling book. Read it.

Ending soundtrack on E6 is Doors’ People are Strange, in Spanish. I like that.

“The preferences that wind up in a mature AI are complicated, practically impossible to predict, and vanishingly unlikely to be aligned with our own, no matter how it was trained.”

And therein lies the problem.

“Once those foundations are in place, though, predicting the outcome of our present trajectory starts to look grimly, horribly straightforward.”