Machine Intelligence Research Institute

The first Dialogue can be considered successful if it establishes that the race to build smarter-than-human general AI systems imposes a wildly unacceptable risk on the entire world. The Dialogue should clearly establish the urgent need for an international agreement banning the development of superintelligent AI (in the spirit of Scher et al., https://arxiv.org/pdf/2511.10783), broadly similar to past bans on chemical and biological weapons development. AI companies have set their sights on building superintelligent AI in the next five years. Researchers are uncertain about the actual timeline, but a large body of experts agree with OpenAI researchers' statement, "Currently, we don't have a solution for steering or controlling a potentially superintelligent AI, and preventing it from going rogue. [...O]ur current alignment techniques will not scale to superintelligence. We need new scientific and technical breakthroughs." Secretary-General Guterres has said, "Alarm bells over the latest form of artificial intelligence, generative AI, are deafening. And they are loudest from the developers who designed it. These scientists and experts have called on the world to act, declaring AI an existential threat to humanity." Modern AI is created by automatic processes, more like "growing" an organism than like crafting a machine. As a consequence, researchers have limited ability to understand and control the systems we're building. So long as AI is weak, this is non-catastrophic; researchers can learn via trial and error. This no longer works once the "errors" become sufficiently dangerous, however. The danger, centrally, is not that bad actors will misuse AI. This is a serious risk, but the more immediate issue is that we simply lack the technical ability to control agentic superhuman AI. No nation or company wins in an AI race; rather, the winner would be the AI itself.

• Safe, secure and trustworthy AI
• Interoperability of governance approaches
• Protection and promotion of human rights
• Transparency, accountability, and human oversight

The selected priorities reflect four interlocking concerns. The central priority is safe, secure, and trustworthy AI. If an AI technology is catastrophically unsafe, its development must be prohibited. Uncertainty about timelines cannot be used as an excuse to delay action: If the international community delays too long, we may lose the opportunity to act at all. Transparency, accountability, and human oversight are operational requirements if the Independent International Scientific Panel can be expected to be informative. International coordination on AI risks cannot proceed without publicly verifiable claims by developers about the capabilities, training, and evaluation of their systems. The building of shared transparency norms - including disclosure of frontier training runs, capability evaluations, deployment scale, and serious incidents - will be among the most consequential contributions the Dialogue can make in its first sessions. Interoperability of governance approaches is necessary because existing instruments have developed largely in parallel and largely without participation from states lacking frontier developers. Interoperability should mean substantive convergence on what is being governed, including frontier capability development. The protection and promotion of human rights provides a normative anchor and continuity with the wider mission of the United Nations. International human rights instruments offer an existing framework for coordinating a response to AI's large-scale potential harms .

The listed themes are broad enough to encompass the key issues, but there is a clear need for more explicit emphasis on smarter-than-human AI and loss-of-control scenarios as a genuinely serious threat. This is not a speculative future concern, but a near-term possibility. The listed themes presuppose that AI is a technology that will always be deployed within and by human-controlled institutions and on a human scale. They do not address the case in which AI systems become capable enough to act in ways their developers cannot reliably predict, audit, or correct. Governance instruments that assume continuous human oversight and reliable interpretability are out-of-step with the actual technical landscape.