Global AI Risks Initiative, Centre for International Governance Innovation (CIGI)

The most important outcome of the first Global Dialogue on AI Governance would be greater awareness of the shared global scale challenges posed by AI and greater understanding of the shared interest in addressing these challenges together. Many public policy issues related to AI can and should be managed at the local or national level or by groups of like-minded countries. Diverse approaches in AI governance respect differences in values and priorities and enable experimentation from which others can learn. However, some challenges posed by AI will be truly global in scale, crossing borders to profoundly affect the lives of citizens regardless of where the AI system was originally developed or deployed. These challenges can only be addressed successfully through international cooperation on AI governance. Global scale challenges will become even more urgent and impactful as AI systems grow more powerful. In particular, the possible future development of artificial superintelligence would have profound and irreversible implications for all humanity. Global AI challenges include: 1. How to prevent to most severe global scale safety and security risks posed by AI, such as catastrophic misuse or loss of human control over autonomous AI systems; 2. How to ensure that the benefits of AI are fully realized and widely enjoyed across the global population; and 3. How to make legitimate decisions on AI issues affecting the future of all humanity, such as whether and under what conditions to create autonomous AI systems vastly more intelligent than humans and/or potentially capable of conscious experience. For more information, see: Framework Convention on Global AI Challenges (CIGI 2024)

Safe, secure and trustworthy AI;

All the themes mentioned above have elements that relate to shared global AI challenges, and that therefore merit attention by the AI Dialogue. However, the first and most urgent priority for the Dialogue should be to address the most severe global scale safety and security risks posed by AI, as maintaining safety and security is a prerequisite for being able to address all other issues. AI is likely to be the most powerful technology ever developed by humans. As such, it has the potential to bring unprecedented benefits in terms of scientific breakthroughs and productivity-enhancing innovation that could drastically grow prosperity and improve human well-being. However, AI also poses unprecedented risks to human safety and security. Some of these risks have the potential to cross borders and affect citizens in all countries, regardless of where the AI is developed or deployed. These global scale risks include misuse of AI to enable widespread cyber or bio-weapons attacks, or human loss of control over autonomous AI systems pursuing interests unaligned with human intentions. Leading AI scientists and technologists believe that AI could even pose an extinction risk to humanity. These risks are likely to grow as AI systems continue to become more capable, and there are signs that the rate of capabilities gain is now accelerating due to increasing AI automation of AI R&D. Citizens in all parts of the world will only be able to enjoy the many potential benefits that AI has to offer if they are simultaneously able to avoid the most severe safety and security risks posed by AI. The global dialogue must help build understanding about the nature of current and future safety and security risks posed by AI, the shared interest of all governments in preventing the most severe global scale manifestations of these risks, and the practical steps that can be taken to build sufficiently safe, secure and trustworthy AI.

A first key cross-cutting issue is to determine which AI challenges are most global in scale and most likely to require international cooperation. As mentioned above, the Global Dialogue should clearly prioritize a focus on shared global scale challenges relating to AI, while leaving other issues to be addressed by nations individually or among groups of like-minded countries. The Global Dialogue must focus on the intersection of all Member State interests, not the sum of their divergent interests on AI. Failing to make this distinction will result in a Global Dialogue that is too diffuse and distracted to make progress on the most important shared challenges requiring urgent coordinated action on AI governance. A second key cross-cutting theme is the need to grapple with high uncertainty. AI in its current form is both relatively new and rapidly evolving, and as a result, there is little that can be reliably extrapolated from past experience. There remains high uncertainty and legitimate debate regarding many key policy-salient cruxes of uncertainty, such as the pace and trajectory of future AI capabilities development, the extent of control or alignment that humans will be able to maintain over future AI systems, and the level of concentration of control over the most capable AI systems. There is also high uncertainty about the speed, shape and scale of potential real world implications of AI, such as for security, employment, productivity, scientific discovery. As a result, the Global Dialogue must avoid the temptation to make hasty assumptions about the most likely future. In the face of high uncertainty and high stakes, responsible public policy making must consider and prepare for a wide range of different possible scenarios, including the most challenging ones. For examples of a scenarios approach to AI governance, see: Special Report: AI National Security Scenarios (CIGI 2026)

AI is a relatively new, rapidly developing and unprecedentedly transformative technology, and governments are notoriously slow to react. Therefore, it is not surprising that there are many governance gaps regarding AI's current and potential impacts across the entire spectrum of policy issues. However, in my view, the most significant governance gap affecting my country (Canada) and all countries is a lack of attention to the most severe national and global security risks posed by current and future AI systems. While governments focus on important priorities for national action, such as increasing AI adoption or responding to the latest specific examples of AI harms such as misinformation or AI-induced suicide, they are mostly failing to address what is likely to have the greatest impact on the lives of their citizens, namely the most advanced AI systems that are being developed, for the most part, outside their borders. Governments are mostly neglecting global scale AI challenges because these issues are neither on their immediate radar nor within their purview to manage at the national level. The Global Dialogue must help to address this gap by strengthening understanding of the global scale challenges posed by AI, building awareness of the shared national interests in addressing such global challenges, and fostering support and momentum for the kinds of internationally coordinated action that will be necessary to meet the shared national self-interest of member states.

The Global Dialogue can advance international cooperation on AI governance first and foremost by focusing attention on the global scale challenges posed by AI and demonstrating to member states where they will only be able to achieve their national self-interest by addressing these global challenges collectively. The Global Dialogue should adopt a global, future-oriented, scenarios-based approach that encourages participants to look beyond their national borders and beyond the current capabilities of AI systems in order to recognize how their national interest and the security of their citizens may be most impacted by near future AI developments outside their borders. The Global Dialogue can help participants to realize that, while most countries (other than the United States and China) cannot directly shape the pace or direction of frontier AI development, that does not mean they can be disengaged. Cooperation among a concerted group of like-minded countries, with the support of a global majority, could be pivotal in encouraging leading AI companies and their host countries to act responsibly on AI development to safeguard their own national interest as well as those of the broader global community.

The Global Dialogue should focus on the most important shared global scale challenges posed by AI, and the need for international coordination and cooperation to address these challenges. As such, while the Dialogue can build on many existing industry, civil society, national and international discussions on AI governance, it should primarily build on initiatives that have made the greatest efforts to focus on shared global AI challenges and how to address them. In the case of addressing the most severe global scale risks of AI, this would include the AI Summit Series (and particularly the side events focused on safety security), the Hiroshima AI Process, the EU AI Act, the network of AI safety and security institutes (and in particular the work of the UK AISI), the International Scientific Report on AI Safety, various state level laws in the United States, work by the frontier AI companies themselves regarding evolving and potential security risks, and work by various independent AI assessment bodies and civil society research organizations such as the International Association for Safe and Ethical AI.

Ideally, the Global Dialogue should be viewed as both the pinnacle of, and the inspiration for, a far larger, deeper, ongoing global dialogue on AI happening both between and within all societies. AI is such a transformative technology that there will be a strong need for rapidly accelerated and intensified dialogue among governments, stakeholders and the public to navigate what is likely to be a tsunami of highly impactful policy decisions. When it comes to the global dialogue itself, and the focus on shared global priorities regarding AI, there would be value in complementing the usual multistakeholder approach with various forms of citizen deliberative dialogues. These could include sustained dialogues among representative citizen participants from different countries. The Global Dialogue should also use scenarios exercises as a key mechanism for stimulating future-oriented big picture understanding and strategic thinking among various stakeholders, including citizens. Scenarios exercises not only help to offset present bias by focusing attention on the future, but they can also help elevate people from beyond parochial concerns and help them to perceive where shared interests may lie.

Certain segments of society are typically underrepresented in public policy discussions, as are certain parts of the world. The Global Dialogue must ensure that it includes a globally diverse and balanced set of participants, and that appropriate steps are taken to ensure that all can participate on an equal basis of knowledge and support. When it comes to the most important global scale challenges posed by AI, the entire broader public is currently underrepresented. This is because governments, for the most part, have so far been reluctant to engage with the kinds of AI issues that could potentially have the greatest impacts on their citizens. The Global Dialogue can play a crucial role in overcoming this deficit by providing a visible forum where governments and stakeholders are asked to address global scale AI challenges explicitly. This can help to legitimize similar conversations at the national level as well.

As mentioned above, the Global Dialogue could use scenarios exercises as a key mechanism for stimulating future-oriented big picture understanding and strategic thinking among various stakeholders, including citizens. Scenarios exercises not only help to offset present bias by focusing attention on the future, but they can also help elevate people beyond their competing interests and help them to perceive where even larger shared interests and priorities may lie. For support in facilitating scenarios exercises on priority global issues, see for example the Strategic Foresight Team at the OECD.

EU: legally binding risk-based governance through EU AI Act; prohibits "unacceptable risk" practices including manipulation causing harm, public-sector social scoring and certain biometric identification; mandates conformity assessments, post-market monitoring and human oversight. South Korea: innovation-led approach through AI Basic Act (2026) with risk-based obligations for "high-impact AI", active international coordination through AI Safety Institute and Seoul Summit. Using key building blocks of AI governance as a comparative framework, the Dialogue can identify which are working in practice, where capacity gaps persist and which safeguards should become interoperable across jurisdictions: https://global-governance.ai/.