The event led to the impactful release of The Singapore Consensus on Global AI Safety Research Priorities, guiding crucial international collaboration. We’re proud to have contributed and look forward to continuing this important work. Read more: aisafetypriorities.org

Read the full paper here: saif.org/research/bar... or on arxiv: arxiv.org/abs/2504.15416

We propose 4 bare minimum safeguards applicable when AI agents significantly automate or accelerate AI development:

2️⃣ AI R&D agents could dramatically accelerate improvement in catastrophic capabilities, leading to adaptation lag for both technical and governance measures as well as dangerous capability proliferation.

We identify two core risk pathways:

1️⃣ As AI agents automate AI research workflows, they could sabotage safety efforts and create an unauthorized internal deployment for dangerous purposes.

In our latest paper, we outline how autonomous AI R&D could lead to serious risks. This work reflects a vital international dialogue, uniting experts across jurisdictions – including authors from China, the US, Canada & Europe – to address these shared global challenges.

Over time, SAIF aims to improve the odds of international AI cooperation. Some other ways we hope to push this is by making policy recommendations, conducting research, and directly facilitating international cooperation ourselves through further convenings of researchers.

To date we've hosted three dialogues, in Oxford, Beijing and Venice. (You can read more about them at idais.ai)

Our foundational program is the
International Dialogues on AI Safety (IDAIS). Output of these dialogues demonstrates consistent scientific consensus on AI risks, and the very existence of these high-profile China/West meetings demonstrates that meaningful international cooperation is possible.