PauseAI Global

A successful first Global Dialogue would establish three foundations. First, shared recognition of the catastrophic and existential risks posed by the race to develop artificial general intelligence (AGI). Frontier AI labs publicly state that they are racing to build systems more capable than humans across virtually all cognitive tasks, with timelines they themselves estimate in years rather than decades. The CEOs of the leading AI companies, alongside hundreds of the field's most cited scientists, have signed statements declaring that "mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." A Dialogue that does not begin from this shared recognition will not be addressing the most consequential governance question of our time. Second, an explicit acknowledgement that existing governance efforts are insufficient. Voluntary commitments from frontier developers have repeatedly been broken. National regulations focus primarily on deployment risks rather than the development of increasingly capable frontier systems. The Dialogue should map these gaps honestly and commit to closing them. Third, a concrete pathway toward binding international coordination on frontier AI development. This is the unique value the UN can provide. Just as the international community developed treaties for nuclear weapons, biological weapons, and ozone-depleting substances, frontier AI requires binding multilateral instruments. A successful first Dialogue would establish a working track toward such an instrument, including: (a) shared definitions of frontier capabilities warranting international oversight, (b) verification and compliance mechanisms drawing on precedents like the IAEA, and (c) a clear timeline for negotiation. Success is not consensus on every issue. Success is the Dialogue establishing itself as the venue where the hardest question, namely how humanity collectively governs the development of systems that may match or exceed human capabilities, is genuinely on the table.

• Safe, secure and trustworthy AI
• Interoperability of governance approaches
• Transparency, accountability, and human oversight
• Social, economic, ethical, cultural, linguistic and technical implications of AI

Safe, secure and trustworthy AI is our highest priority because the central governance challenge of our time is the uncontrolled race to develop artificial general intelligence. Frontier developers themselves acknowledge that the alignment problem, namely ensuring advanced AI systems pursue intended goals without catastrophic side effects, is unsolved. Deploying systems whose behaviour we cannot reliably predict or control, at capability levels approaching or exceeding human performance, represents an unprecedented risk to international peace and security. The UN must address this directly. Interoperability of governance approaches matters because frontier AI development is a coordination problem. Unilateral national restrictions are undermined by jurisdictional arbitrage. Without binding international coordination on the most capable systems, every government faces pressure to deregulate to avoid losing the race. Only a multilateral framework can break this dynamic. The Dialogue is uniquely positioned to lay the groundwork for such a framework, drawing on precedents from arms control, biosafety and environmental treaties. Transparency, accountability, and human oversight are foundational. Frontier AI systems are currently developed behind closed doors, with limited external scrutiny of their capabilities, training processes, or safety evaluations. Governments and the public lack the information needed to make informed decisions about acceptable risk. International requirements for capability disclosure, third-party auditing, incident reporting, and meaningful human control over consequential decisions are essential preconditions for any other governance effort to succeed. Social, economic, ethical, cultural, linguistic and technical implications of AI is critical because the economic disruption already underway, particularly the rapid automation of cognitive labour, threatens to concentrate wealth and power, destabilise democracies, and erode the social fabric that any governance regime depends on. Addressing AGI risks requires also addressing the economic transformation happening now, which is rapidly reshaping the political conditions under which governance decisions will be made.

Three cross-cutting issues warrant explicit attention. First, the development of frontier AI systems as a category distinct from AI deployment. Most existing governance frameworks focus on how AI is used. The greater risks come from what is being built: increasingly general, increasingly autonomous systems whose capabilities are not fully understood even by their developers. The Dialogue must distinguish frontier development from narrow application, and recognise that the former requires its own governance regime, including thresholds above which international oversight becomes mandatory. Second, the option of pausing or restricting the development of frontier AI systems beyond defined capability thresholds. Current discourse treats continued capability scaling as inevitable. It is not. The international community has repeatedly chosen to restrict or prohibit technologies whose risks outweighed their benefits, including biological weapons, human cloning, and certain classes of autonomous weapons. A Global Dialogue that does not include the option of binding restrictions on frontier development is foreclosing the most direct response to the most serious risk. We urge the Dialogue to treat a coordinated international pause on frontier AI development beyond agreed capability thresholds as a credible policy option, not a fringe position. Third, democratic legitimacy and meaningful public participation. Decisions about AI development are currently being made by a small number of private actors and the governments closest to them. The trajectory of a technology that may reshape every aspect of human life cannot legitimately be set by this narrow group. The Dialogue should commit to mechanisms that bring affected communities, workers, civil society, and the global public into substantive decision-making, not merely consultative roles. Citizens' assemblies, structured public deliberation, and binding civil society representation in governance bodies are concrete tools the UN should champion. These three issues, namely frontier development as a distinct category, pause as a credible option, and democratic legitimacy, cut across all four thematic clusters.

PauseAI is a global federation of civil society organisations operating in over 15 countries, representing citizens concerned about catastrophic and existential risks from frontier AI development. Here is how the governance gaps in the thematic areas we selected affect us. On safety and trustworthiness: Frontier AI labs are now using their own models to accelerate development of the next generation, with credible reports of recursive self-improvement loops shortening already short timelines. Our members include researchers, parents, students and workers who recognise that no government has the visibility, let alone the authority, to determine whether the next generation of frontier systems can be developed safely. This is not a future concern. Models being trained today will be deployed within 6-12 months under governance frameworks designed for a previous generation of technology. On interoperability: Our chapters in the EU, UK, Canada, Kenya, Germany, France, Czech Republic and elsewhere observe firsthand how jurisdictional fragmentation undermines national efforts. Developers headquartered in lightly regulated jurisdictions release systems that affect citizens everywhere. National governments tell our advocates privately that they cannot act alone without disadvantaging their domestic industries. This coordination failure is the central obstacle to meaningful governance, and it can only be resolved multilaterally. On transparency and oversight: Our volunteers, including AI researchers and former employees of frontier labs, repeatedly encounter the same problem: the information needed to assess risk is held privately by developers, with limited disclosure to even the most capable national regulators. Civil society and the public are excluded entirely. On economic and social implications: Our chapters are increasingly engaged with unions, student organisations and worker associations whose constituencies are experiencing rapid AI-driven displacement. The opportunity is significant. These communities are mobilising and seek a constructive governance pathway. The risk is that without one, public concern consolidates into reactive backlash rather than informed participation.

The AI Dialogue can play a role no other forum currently fills: serving as the universal, legitimate venue where binding international coordination on frontier AI development is genuinely on the table. Establishing shared facts. Existing forums often operate from divergent assumptions about AI capabilities, timelines and risks. The Dialogue can convene authoritative scientific assessments, drawing on the model of the IPCC, that establish a shared evidence base on frontier AI capabilities, risks and governance options. The International AI Safety Report led by Yoshua Bengio is a useful starting point that the UN should institutionalise and expand. Legitimising the full range of policy options. Current international discourse treats continued frontier capability scaling as inevitable. The Dialogue can change this by treating the full range of governance options as legitimate subjects of negotiation, including capability thresholds above which development requires international authorisation, compute governance, and coordinated pauses on specific classes of development. The UN's unique convening authority makes it the only body that can credibly place these options on the global agenda. Building toward binding instruments. Voluntary commitments from frontier developers have repeatedly proven insufficient. The Dialogue should establish a clear pathway from deliberation toward binding multilateral instruments, drawing on precedents including the Treaty on the Non-Proliferation of Nuclear Weapons, the Biological Weapons Convention, and the Montreal Protocol. Each of these emerged from sustained international dialogue that produced real obligations, verification mechanisms and institutional capacity. Ensuring universal participation. Frontier AI is currently being developed in a small number of jurisdictions, with consequences for every country on Earth. The Dialogue is the only forum where every state, regardless of its current AI capacity, has standing to participate in shaping the rules. This legitimacy is not symbolic. It is the precondition for any governance regime that will hold over time.

The Dialogue should connect with several existing initiatives while providing distinctive added value. Scientific assessment infrastructure. The International AI Safety Report, produced by an independent expert panel chaired by Yoshua Bengio with contributions from 30+ countries, provides a strong foundation for shared scientific understanding. The Dialogue should institutionalise this report under UN auspices, giving it the authority and continuity needed to inform negotiations. National and regional governance frameworks. The EU AI Act, the UK and US AI Safety Institutes, the Bletchley and Seoul Declarations, and the Council of Europe Framework Convention on AI represent the most developed governance work to date. The Dialogue should map these frameworks for compatibility and gaps, identifying where international harmonisation is achievable and where the absence of binding multilateral instruments leaves critical risks ungoverned. The AI Safety Institute network. The emerging network of national AI Safety Institutes provides important technical capacity for evaluation and red-teaming. The Dialogue should support its expansion to all interested member states and connect it to the UN system, ensuring evaluation capacity is not concentrated in a handful of jurisdictions. Civil society and affected communities. Networks including PauseAI, the Future of Life Institute, the Centre for AI Safety, professional bodies, labour organisations, and academic communities represent constituencies directly engaged with AI risks. The Dialogue should establish formal mechanisms for substantive civil society participation. The Dialogue's distinctive added value is universal legitimacy, the authority to address frontier development as a distinct governance category, and the mandate to move from voluntary coordination toward binding multilateral instruments. No other body combines these three.

The Dialogue should distinguish between consultative participation and substantive participation, and ensure both are meaningful. Member states should hold decision-making authority, consistent with the UN's intergovernmental character. Frontier AI developers should participate, but with clear acknowledgement that they have direct commercial interests in the outcome. Their input should be transparent, on the record, and balanced by equivalent representation from independent scientific and civil society voices. The model of industry self-governance has been tried and has produced insufficient results. Whistleblowers and former employees of frontier AI developers require dedicated channels and protections. A growing number of researchers and engineers who have worked at the frontier of AI development have left their organisations citing serious safety concerns. Their firsthand knowledge of internal practices, safety cultures and capabilities is uniquely valuable and unavailable through any other source. The Dialogue should establish formal mechanisms for their testimony, with appropriate legal protections against retaliation, non-disparagement clauses and other instruments currently used to silence them. Independent technical experts, particularly those without financial ties to frontier developers, should have standing equivalent to industry representatives. The International AI Safety Report panel offers a useful template. Civil society, including organisations representing workers, students, affected communities and citizens concerned about catastrophic risks, requires formal mechanisms for substantive input, not only consultative roles. The structure should include both plenary deliberation and thematic working groups producing concrete deliverables on a defined timeline.

Several constituencies are systematically underrepresented in current AI governance discussions. Affected workers and communities. The people whose livelihoods, public services and information environments are being reshaped by AI deployment are largely absent from the rooms where governance decisions are made. Trade unions, professional associations, teachers, healthcare workers, creative workers and others experiencing AI-driven changes firsthand have direct knowledge that current discussions lack. The Dialogue should establish formal mechanisms for their participation. Citizens concerned about catastrophic and existential risks. A significant and growing share of the global public, reflected in polling across multiple countries, expresses serious concern about the trajectory of frontier AI development. This constituency is currently represented almost exclusively by a small number of expert organisations and is largely absent from intergovernmental forums. The Dialogue should treat public concern about catastrophic risks as a legitimate input to governance, not a fringe view to be managed. Future generations. AI governance decisions made now will shape the conditions under which all subsequent generations live. No current institutional mechanism represents their interests. The Dialogue should consider formal mechanisms for representing future generations, drawing on precedents including Wales's Future Generations Commissioner and the UN Secretary-General's proposed Envoy for Future Generations. Independent scientific voices. Researchers without commercial ties to frontier developers, particularly those raising concerns about catastrophic risks, are often outweighed in current forums by industry-affiliated experts. The Dialogue should correct this imbalance.

Honest assessment: This question is not central to PauseAI's mission, and we have no distinctive expertise on UN process design.

Several existing policies and approaches offer useful foundations, though we note honestly that none yet adequately addresses frontier AI development. Capability thresholds and compute governance. The EU AI Act's designation of "general-purpose AI models with systemic risk" above defined compute thresholds, and similar provisions in the US Executive Order 14110 (since rescinded), establish the principle that frontier systems warrant different treatment based on capability. This principle is sound; the thresholds, enforcement mechanisms and international coordination remain inadequate. Pre-deployment evaluation and red-teaming. The UK and US AI Safety Institutes' work evaluating frontier models before deployment, including the agreements with leading developers to test models pre-release, establishes important precedent. This capacity should be expanded internationally and made mandatory rather than voluntary. Whistleblower protections. California SB 53 (Artificial intelligence models: large developers, 2025) establishes whistleblower protections specifically for frontier AI employees. This is the first law of its kind globally and offers a template for international adoption. Treaty precedents for binding restrictions. The Treaty on the Non-Proliferation of Nuclear Weapons, the Biological Weapons Convention, the Chemical Weapons Convention, and the Montreal Protocol all demonstrate that the international community can negotiate binding restrictions on dangerous technologies, with verification mechanisms, when the stakes warrant it. The IAEA's verification regime for nuclear materials offers a particularly relevant institutional model for compute governance. Citizens' assemblies on technology governance. France's Convention Citoyenne pour le Climat and Ireland's citizens' assemblies demonstrate that structured public deliberation can produce considered judgement on complex governance trade-offs. What is currently missing. No existing initiative establishes binding international restrictions on the development of frontier AI systems beyond defined capability thresholds. This is the central governance gap. The Dialogue's most important contribution would be to begin closing it.