Center for Existential Safety

The Global Dialogue on AI Governance would be a success if it resulted in a widely adopted Collective Commitment to AI Safety Statement: "We declare humanity's collective commitment to immediately mitigate the existential, catastrophic, and astronomical suffering risks posed by the development of artificial superintelligence." Furthermore, the Dialogue should also include a full endorsement of the Statement on Superintelligence (superintelligence-statement.org) by the UN System: "We call for a prohibition on the development of superintelligence, not lifted before there is 1. broad scientific consensus that it will be done safely and controllably, and 2. strong public buy-in."

Safe, secure and trustworthy AI

The advancement of frontier AI introduces immediate risks of human extinction, civilizational collapse, and astronomical levels of suffering. Failure to effectively manage these risks will likely create unprecedented levels of harm for all moral patients on Earth.

We think our primary focus must first be on mitigating the existential, catastrophic, and astronomical suffering risks of frontier AI.

The lack of a supranational body dedicated to global governance of AI puts all life on Earth at extreme risk. It is a moral imperative of the highest order to rectify that immediately. We call on the UN to become that body.

We urge the AI Dialogue to advocate for the UN Independent International Scientific Panel on AI to take a direct role in global governance of AI research and development. It should use every resource the UN System has available to facilitate international cooperation on AI. If it can't, then we encourage the AI Dialogue to result in a call for the formation of a new body to serve this purpose. We propose the International AI Governance Alliance (iaiga.org), a new supranational organization dedicated to effectively mitigating extinction risks from AI and fairly distributing its economic benefits to all.

We encourage the AI Dialogue to directly interface with the many hundreds of organizations in the AI safety space (aisafety.com/map), especially with the International Association of Safe and Ethical AI (iaseai.org). The legitimacy of the AI Dialogue with the expertise of the organizations directly tackling the risks and opportunities inherent in AI is a potent force for good. Consider inviting representatives from the largest 100 AI safety organizations to all in person meetings.

We suggest all stakeholders are invited to participate in meetings either virtually or in person. In person attendees ought to be decided primarily based on these two criteria: 80%: Number of lifetime hours dedicated to existential safety and/or global governance 20%: Potential for unique perspectives or contributions to the proceedings

The most underrepresented voices are from those with the most time dedicated to AI safety. In other words, the originators of the AI safety movement: Eliezer Yudkowsky, Nate Soares, Luke Muehlhauser, Nick Bostrom, and several others. For example, few members of the current Independent International Scientific Panel on AI have more than 10+ years of experience in AI safety. These individuals do. Consider inviting them as advisors.

We suggest implementing a futarchy-based (en.wikipedia.org/wiki/Futarchy) voting system whereby the public and experts could both suggest policies and have them evaluated in public prediction markets.

With our sister organization, the International AI Governance Alliance (iaiga.org), we are experimenting with a number of governance mechanisms. When we have finalized our current set of governance mechanisms, we will post them on the website.