Lots to like in this piece. Still, I don't think Narayanan and Kapoor address what's unusual unusual about AI versus past general-purpose tech—even if it mostly behaves like “normal” technology knightcolumbia.org/content/ai-a...

This is a rarity in frontier AI discourse: Bengio et al. endorse the precautionary principle—arguing we must prove safety before scaling.

Their “Scientist AI” proposal allows us to disable agentic and planning components—building in off-switches from the start.

📄 arxiv.org/abs/2405.20009 #bluesky

Some great initiatives for tracking AI harms I've been following so far include:

- AIAAIC (www.aiaaic.org/aiaaic-repos...) and
- MIT's AI Incident Tracker (airisk.mit.edu/ai-incident-...).

Pretty shocking to see the numbers on autonomous vehicle incidents. Very few of these reach the headlines.

Not all x-risk bangs. Some simmer. Kasirzadeh warns: AI may collapse social systems via accumulative harms—slow-moving, systemic, invisible. Real systems unravel through misalignments over time.

AI safety needs tools to track compound harm.

📑 arxiv.org/abs/2401.07836

#TechEthics #bluesky

Most model evals focus on benchmarks—but what about catastrophic misuse? Shevlane et al. propose tools for extreme risk evals, urging labs to test frontier AI models for deception, persuasion, and autonomy before deployment.

To what extent is this happening in practice?

📄 arxiv.org/abs/2305.15324

What if existential risk from AI doesn’t arrive with a bang, but builds slowly beneath our feet? Kasirzadeh warns of a ‘boiling frog’ scenario—AI risks that compound silently, eroding systems until collapse. We must reckon with both the decisive and the accumulative 💭

📄 arxiv.org/abs/2401.07836

What I find useful in Peterson’s approach is how it sidesteps the usual “which theory is right?” trap. Instead of starting with utilitarianism or deontology, he looks at recurring judgment patterns. That kind of mid-level mapping seems especially helpful in bio risk, where stakes are so high

1/9 Right after landing back in San Francisco, I was greeted by the billboard pictured below: “Win the AGI Race.” I had just returned from the KU Leuven's conference on Large-Scale AI Risks, where we spent several days in serious conversation about the long-term consequences of advanced AI systems.

Back from KU Leuven's AI risk conf, greeted by ths billboard: “Win the AGI Race.” After days discussing disempowerment—the slow loss of human influence from AI—this felt ominous. If 'winning' = automating all work, what’s left? Safety must outrun speed. 📄 arxiv.org/abs/2501.16946 #AI #AISafety #tech

Presented my AI safety work at KU Leuven’s International Conference on Large-Scale AI Risks. Both terrifying & exciting to be on the same schedule as people I’ve been reading for years! The tone throughout was frank, serious, and grounded in hard questions.

Did you know that current EV battery technology uses the same chemicals as non-stick Teflon pans? Most of these PFAS-containing components will end up in landfills, only to leach into the communities surrounding them. But which communities? open.substack.com/pub/amplifie...