Korea AI Safety Institute

The success of the first Global Dialogue should be assessed not by the articulation of general principles, but by whether it establishes the minimum conditions for governing risks that may become irreversible beyond certain thresholds. Some advanced AI risks exhibit tipping point dynamics: once a certain level of capability or deployment is reached, mitigation through ex post regulation may no longer be feasible. In such contexts, governance cannot rely on incremental national approaches alone. Competitive technological environments create incentive structures analogous to a prisoner's dilemma, in which individual actors—states or firms—may benefit from deviating from safety constraints, even when collective adherence would produce better outcomes. A successful Dialogue should therefore achieve three outcomes. First, it should advance a shared understanding of risk thresholds that warrant preventive limitation or prohibition. Second, it should recognize the need for governance arrangements that reduce incentives for strategic defection, including forms of mutual constraint and interdependence. Third, it should initiate discussion on operational mechanisms—such as independent evaluation, incident reporting, and controlled access to high-risk systems—that move beyond declaratory commitments. Ultimately, success lies in shifting the conversation from voluntary alignment to governance structures in which deviation from agreed safeguards is systematically constrained.

• Safe, secure and trustworthy AI
• Interoperability of governance approaches
• Protection and promotion of human rights
• Transparency, accountability, and human oversight

These four priorities reflect the need to integrate technical safety, institutional coordination, and normative legitimacy in AI governance. Safe, secure and trustworthy AI provides the technical foundation for preventing high-impact and potentially irreversible failures. However, technical measures alone are insufficient in a fragmented regulatory landscape. Interoperability of governance approaches is therefore essential to enable alignment across diverse national systems without requiring full harmonization. Transparency, accountability, and human oversight address a central weakness in current governance frameworks. As AI systems become more autonomous and capable, human involvement risks becoming procedural rather than substantive. Ensuring that oversight remains meaningful in practice-not merely formal-is critical for maintaining control and accountability. Protection and promotion of human rights provides the normative basis for governance. Yet, in practice, these principles are often challenged by competing priorities such as security and economic competition. Strengthening their operational relevance is therefore essential. Taken together, these priorities support a governance approach that is technically grounded, internationally coordinated, and normatively robust, while also addressing the risk of formal compliance without effective control.

Several cross-cutting issues are not fully captured in the listed themes. First, the governance of irreversible or tipping-point risks requires explicit attention. Certain AI capabilities may generate consequences that cannot be effectively mitigated once deployed, making preventive governance and threshold-based approaches essential. Second, the strategic incentive structure of AI development resembles a prisoner's dilemma. States and firms may face pressures to deviate from safety constraints in order to maintain competitive advantage. Without mechanisms that reduce or counterbalance these incentives, voluntary compliance is unlikely to be sufficient. Third, there is a structural limitation in independent evaluation. Frontier AI systems are concentrated among a small number of actors who control the computational resources and access required for meaningful assessment. This creates dependence on developer cooperation and undermines the feasibility of external oversight. Fourth, the use of security and public safety exemptions in existing governance frameworks risks weakening the credibility of regulatory prohibitions. Such exemptions can enable the very practices that are formally restricted, both domestically and through international diffusion. Finally, the concept of meaningful human control requires reassessment. In systems where decision-making and justification are increasingly automated, human oversight may become nominal, raising questions about accountability and responsibility. Addressing these issues requires moving beyond principle-based governance toward structures capable of managing strategic behavior and ensuring enforceability.

The current governance gaps are already producing tangible effects in technologically advanced, highly networked economies such as South Korea, particularly in the areas of AI safety, public governance, and critical infrastructure. One significant challenge is the growing asymmetry between national regulatory capacity and the concentration of frontier AI capabilities in a small number of global firms. Public institutions and independent researchers face structural limitations in accessing and evaluating advanced systems, which constrains the ability to conduct meaningful risk assessments or enforce safety standards. This creates a dependency on external providers that is difficult to reconcile with national accountability requirements. A second challenge concerns the expanding use of security and public safety exceptions. While regulatory frameworks formally emphasize human rights and responsible AI, these exceptions introduce ambiguity in implementation and risk undermining public trust. This is particularly relevant in areas such as surveillance, data governance, and national security applications. A third issue is the increasing gap between formal human oversight and operational reality. As AI systems become embedded in decision-support processes across sectors, including public administration and security, the pace and scale of automated outputs can exceed the capacity for substantive human review. At the same time, there are important opportunities. Countries with advanced digital infrastructure and regulatory institutions are well positioned to contribute to demand-side governance by setting market-based safety requirements and certification standards. In addition, middle-power countries can play a bridging role in promoting interoperability across governance systems and facilitating coordination between major technological actors. These dynamics underscore the need for governance approaches that address structural constraints on oversight, align incentives across jurisdictions, and ensure that safety mechanisms remain effective in practice.

The AI Dialogue can play a critical role by shifting international cooperation from principle-based alignment to coordination under conditions of strategic interdependence. Current AI governance efforts often assume that voluntary alignment around shared principles will be sufficient. However, the development and deployment of advanced AI systems increasingly take place in competitive environments characterized by strong incentives to defect from safety commitments. In this context, the Dialogue can serve as a platform to explicitly recognize and address these structural dynamics. First, it can facilitate convergence on risk thresholds that require preventive action, particularly in areas where tipping-point dynamics may lead to irreversible outcomes. Establishing shared understandings of such thresholds is a prerequisite for meaningful cooperation. Second, the Dialogue can support the development of interoperable governance mechanisms that reduce incentives for unilateral deviation. This includes coordination on incident reporting, evaluation standards, and conditions for access to high-risk systems. Third, it can provide a structured space for bridging different governance approaches, including supply-side and demand-side models. By bringing together states with varying capabilities and roles in the AI ecosystem, the Dialogue can help align incentives across producers, deployers, and regulators. Finally, the Dialogue can strengthen the credibility of international governance by addressing gaps between formal commitments and operational realities. By focusing on enforceability, oversight capacity, and accountability, it can help ensure that cooperation translates into effective risk management. In this sense, the Dialogue's primary role is not only to enable discussion, but to anchor the transition toward governance frameworks capable of functioning under conditions of competition and uncertainty.

The AI Dialogue should build upon and connect existing initiatives across three main domains: international political processes, technical safety frameworks, and sector-specific governance mechanisms. First, it should link to recent multilateral processes such as the Bletchley, Seoul, and Paris AI Summits, which have contributed to a growing convergence in the identification of high-impact risks and the need for coordinated responses. These processes provide a foundation for shared risk framing, but have not yet translated into operational mechanisms. Second, the Dialogue should engage with emerging technical governance efforts, including model evaluation protocols, red-teaming practices, and safety standards developed by research institutions and industry. These initiatives offer practical tools but remain fragmented and unevenly accessible. Third, it should draw lessons from established governance regimes in other high-risk domains, such as nuclear safety, aviation incident reporting, and cybersecurity information sharing. These fields demonstrate the importance of structured reporting systems, independent oversight, and mechanisms for collective response. The added value of the AI Dialogue lies in its ability to connect these fragmented efforts into a coherent international framework. Unlike existing initiatives, it operates within the United Nations system, allowing for broader participation and greater legitimacy. More importantly, the Dialogue can serve as a coordination layer that aligns political commitments with technical practices and economic incentives. By integrating supply-side and demand-side approaches, and by addressing issues of enforceability and strategic behavior, it can move global AI governance beyond voluntary cooperation toward more resilient and effective forms of coordination.

Different stakeholders contribute to AI governance through distinct forms of authority—technical expertise, regulatory capacity, market influence, and normative legitimacy. The AI Dialogue should be structured to reflect and integrate these differentiated roles. Governments can contribute by articulating regulatory priorities and identifying areas where international coordination is necessary, particularly in relation to risk thresholds and enforcement challenges. Industry actors can provide insight into technical feasibility, system capabilities, and constraints in implementation. Academic and independent research communities are essential for advancing methodologies for evaluation, risk assessment, and oversight. Civil society can highlight societal impacts, human rights concerns, and accountability gaps. To be effective, the Dialogue should move beyond general plenary discussions and adopt a layered structure. First, thematic working groups should focus on specific governance challenges, such as risk thresholds, evaluation access, and incident reporting. Second, cross-stakeholder sessions should be designed to address areas of strategic tension, including trade-offs between safety, innovation, and security. Third, outputs from these discussions should be consolidated into actionable recommendations with clear follow-up mechanisms. Importantly, the structure should ensure continuity across annual Dialogues, including tracking progress on agreed priorities and identifying areas where cooperation remains insufficient. Such a format would enable the Dialogue to function not only as a forum for exchange, but as a mechanism for progressively building interoperable and enforceable governance practices.

Several perspectives remain underrepresented in global AI governance discussions, particularly those most directly affected by the deployment of high-risk systems. First, middle-power and technologically capable countries outside the core group of frontier AI developers are often underrepresented. These countries play critical roles as regulators, adopters, and market shapers, yet their perspectives on implementation and demand-side governance are insufficiently reflected. Second, independent researchers and public-interest institutions face structural barriers to participation. Limited access to frontier systems constrains their ability to contribute to evaluation and oversight discussions, which in turn affects the diversity and credibility of governance debates. Third, communities directly exposed to AI risks—such as civil society actors, journalists, and vulnerable populations—remain insufficiently included, particularly in discussions shaped by security or technical expertise. To address these gaps, the Dialogue should institutionalize mechanisms for inclusion. This includes targeted support for participation from underrepresented regions, structured input channels for independent researchers, and dedicated sessions focused on societal impact and lived experience. In addition, improving access to evaluation resources and data would enable broader and more meaningful participation in technical governance processes. Inclusion should not be limited to representation, but should extend to meaningful influence over agenda-setting and outcomes.

Innovative engagement formats should be designed to address the complexity and strategic nature of AI governance challenges. One effective approach is the use of scenario-based simulations. These exercises can model high-risk or tipping-point situations—such as systemic failures or cross-border incidents—and allow stakeholders to explore coordinated responses under realistic constraints. Such formats can reveal gaps in existing governance mechanisms and clarify areas requiring cooperation. A second approach is structured negotiation exercises focused on specific governance dilemmas, including risk thresholds, access to evaluation, and the management of dual-use technologies. By simulating decision-making under conditions of strategic competition, these formats can help identify feasible pathways for agreement. Third, cross-disciplinary "red-teaming" sessions can be introduced to stress-test governance proposals. Participants from technical, legal, and policy backgrounds can jointly evaluate whether proposed frameworks remain effective under adversarial or high-pressure conditions. Finally, the Dialogue could incorporate iterative working formats that extend beyond the event itself, such as ongoing expert groups or pilot initiatives. This would ensure that engagement is not limited to discussion but contributes to the development of operational solutions. These formats would enable the Dialogue to move from static exchange toward dynamic problem-solving, better reflecting the evolving nature of AI risks and governance challenges.

Several emerging policies and institutional approaches provide useful building blocks for effective AI governance, particularly where they combine regulatory design with operational capacity. First, regulatory frameworks are evolving in different but partially convergent directions. The European Union's AI Act adopts a risk classification approach, defining categories such as high-risk systems and general-purpose AI (GPAI) models, with differentiated obligations. In contrast, South Korea's AI Basic Act focuses more directly on high-impact systems and frontier models, reflecting a policy orientation toward managing advanced capabilities and systemic risks. While both approaches aim to prioritize more consequential AI systems, they are not fully equivalent. This diversity underscores the importance of interoperability rather than strict harmonization. Second, institutional models for AI safety oversight are becoming increasingly important. The establishment of the AI Safety Institute (AISI) in South Korea represents a meaningful step toward building dedicated capacity for risk evaluation, policy coordination, and international cooperation. Such institutions can help bridge the gap between high-level principles and operational implementation, particularly in areas requiring continuous technical assessment. Third, pre-deployment evaluation practices-including red-teaming, model testing, and capability assessments-developed by leading AI developers and research communities provide concrete methodologies for identifying risks before deployment. These practices are increasingly being referenced in national and international governance discussions. Fourth, a distinctive strength of the AI safety field is the presence of well-resourced non-governmental actors, including philanthropic foundations and independent research organizations, that actively contribute to safety research, evaluation methods, and policy development. Their financial and technical capacity enables rapid experimentation and cross-border collaboration, often complementing slower governmental processes. Fifth, emerging discussions on AI incident reporting mechanisms, inspired by aviation safety and cybersecurity, offer a promising model for collective risk management through shared learning and early warning systems. In parallel, demand-side governance approaches-such as public procurement standards, certification schemes, and liability frameworks-are gaining relevance as tools for shaping incentives in global AI deployment. At the same time, these initiatives face structural limitations, including restricted access to frontier systems and uneven enforcement. Their effectiveness will depend on integration into broader international frameworks that address independent evaluation, interoperability, and alignment of incentives across jurisdictions.