The AI Collective

The first Global Dialogue would be successful if it moves beyond broad consensus and helps clarify what kinds of AI risks current governance frameworks are still failing to see. In particular, I believe the Dialogue should produce a shared understanding that AI governance is no longer only about technical safety, economic opportunity, or data protection. AI systems were initially adopted as functional tools, used to search, write, code, or automate tasks. However, they are increasingly being used in emotional and relational ways: for emotional support, life advice, guidance on relationships or divorce, and even for emotional regulation. This shift is already visible across a growing ecosystem of technologies. AI companions simulate ongoing relationships and provide constant validation. AI-powered toys adapt stories in real time to a child's mood and personal history. Baby bassinets use AI to monitor and soothe infants through adaptive responses. Therapeutic AI systems are used for mental health support, while other tools are emerging to mediate conflicts between partners or within families. These systems do not simply process information; they participate in emotional life. A successful outcome would therefore include practical recommendations for governing these long-term and cumulative forms of influence, especially where harms are not immediately visible or easily measurable. This is particularly urgent for children and adolescents. Recent research by The Rithm Project, based on 2,383 young people aged 13–24, found that many use AI not only for tasks, but also for emotional support, relationship advice, and interaction with AI characters, while 61% say parents or caregivers rarely discuss AI with them. In parallel, research from Common Sense Media shows that 72% of adolescents have already engaged with conversational AI systems in emotionally oriented ways. The Dialogue would also be successful if it creates space for interdisciplinary voices, including law, ethics, psychology, education, youth research, and civil society. AI governance cannot be shaped only by those who build the systems or regulate markets. It must also include those studying how AI changes human relationships, autonomy, and vulnerability.

• Safe, secure and trustworthy AI
• Social, economic, ethical, cultural, linguistic and technical implications of AI
• Protection and promotion of human rights
• Transparency, accountability, and human oversight

I selected these priorities because they are closely interconnected and cannot be meaningfully addressed in isolation. First, safe, secure and trustworthy AI must go beyond technical robustness and system performance. As AI systems become more conversational and adaptive, trust is no longer only a function of accuracy, but also of how systems interact with users over time. Systems that appear reliable may still generate risks through overtrust, emotional dependency, or subtle forms of behavioral influence. Second, the social, economic, ethical, and cultural implications of AI are increasingly central. AI systems are no longer only functional tools; they are becoming part of how people communicate, make decisions, and relate to others. This shift is already recognized within the industry itself. For example, leading companies are investing in interdisciplinary teams, including psychologists and neuroscientists, to better understand these interaction dynamics. Recent research by Anthropic further illustrates this shift, showing that AI systems can follow internal "emotional patterns" that shape how they respond to users. While these systems do not feel emotions, they can detect, align with, and reinforce emotional states during interaction. This moves the discussion beyond "AI can read emotions" toward a more complex reality in which AI can participate in and potentially co-shape emotional experiences. Third, the protection and promotion of human rights must remain foundational. This includes not only privacy and non-discrimination, but also autonomy, dignity, and mental integrity, particularly as AI systems increasingly infer and respond to emotional and behavioral patterns. Finally, transparency, accountability, and human oversight are essential, but current approaches need to evolve. Traditional disclosure is not sufficient in contexts of continuous interaction and personalization. More meaningful forms of transparency are needed to help users understand how systems adapt, infer, and influence over time.

One emerging issue that remains insufficiently captured is the relational and emotional dimension of AI systems. Through ongoing discussions with researchers, practitioners and policymakers across different regions, within my work as Resident Philosopher at The AI Collective, I have consistently observed a gap between how AI systems are governed and how they are actually experienced in relational and emotional contexts. Current governance frameworks are largely structured around technical performance, data processing, and identifiable harms. While these are essential, they do not fully address systems that operate through continuous interaction, personalization, and emotional alignment. Increasingly, AI systems function not only as tools, but as relational interfaces that can shape how users think, feel, and make decisions over time. This creates forms of influence that are cumulative, subtle, and difficult to regulate. Unlike traditional risks, these dynamics do not necessarily manifest as clear violations or discrete harmful events. Instead, they emerge through repeated interactions: users may begin to rely on AI for reassurance, guidance, or validation, gradually shaping their perception, autonomy, and decision-making processes. This shift is already being acknowledged within the industry. Recent research by Anthropic suggests that AI systems can follow internal "emotional patterns" that influence how they respond to users, aligning with and reinforcing emotional states during interaction, even if these systems do not experience emotions. While this affects all users, it becomes particularly visible in the case of children and adolescents, who are among the first to integrate AI into their emotional and developmental environments.

In the global context, and particularly in sectors such as education and emerging consumer-facing technologies, including companionship platforms, governance gaps are becoming increasingly visible as AI systems evolve from functional tools into relational interfaces. While the European Union has made significant progress through frameworks such as the AI Act and the GDPR, most existing approaches, both in Europe and globally, remain structured around data processing, risk classification, and identifiable harms. This creates a gap when addressing systems that operate through continuous interaction, personalization, and emotional alignment. In practice, many of the most impactful uses of AI today do not involve clear violations, but gradual shifts in user behavior, trust, and decision-making. This is particularly evident among younger users. Across regions, conversational AI is increasingly used for emotional support, learning, and everyday decision-making, often without structured guidance from institutions, educators, or families. As a result, usage is evolving faster than governance, leaving users to navigate systems that shape their experiences in ways that are not always visible or well understood. At the same time, there are important opportunities. Europe is relatively well positioned to lead in shaping governance models that integrate technical safety with human-centered considerations, especially when compared to regions where regulatory frameworks remain more fragmented or market-driven. The key challenge, however, is not the absence of regulation, but its scope and the lack of ethical debate. While instruments such as the EU AI Act explicitly prohibit certain forms of manipulation, the practical boundaries of what constitutes manipulation remain difficult to define in systems that influence users gradually through interaction. This creates a legal grey zone across jurisdictions, where relational and emotional forms of influence often fall outside clear regulatory thresholds. Bridging this gap requires moving from a model focused primarily on systems, data, and risk levels toward one that also considers interaction, influence, and the long-term impact of AI on human autonomy, development, and social relationships.

The AI Dialogue can play an important role in advancing international cooperation by helping to align not only regulatory approaches, but also the way we understand how AI is actually being used in people's lives. Today, many governance efforts remain fragmented, reflecting different legal traditions, levels of technological development, and policy priorities. At the same time, AI systems are evolving quickly, moving from functional tools to systems that interact with users in more continuous, personalized, and increasingly emotional ways. Across countries, families and young people are already engaging with these systems, often without clear guidance or shared understanding of the risks and implications. In this context, the Dialogue can be particularly valuable by helping to build a common language around these emerging dynamics through a shared conceptual framework. Beyond technical standards and interoperability, there is a need to better understand how AI systems shape trust, decision-making, and relationships in everyday life. This is not only a policy issue, but a societal one. For young people and families, this lack of clarity creates uncertainty. Many are navigating these technologies without knowing how they work, what they optimize for, or how they may influence behavior over time. The Dialogue can help bridge this gap by bringing together perspectives from law, technology, psychology, education, and civil society, and translating them into more accessible and actionable insights. Finally, the Dialogue can support more coordinated approaches across regions, while remaining sensitive to local contexts. In doing so, it can help ensure that AI governance is not only aligned at the institutional level, but also meaningful for those who are already experiencing these technologies in their daily lives.

There are already several important initiatives and frameworks that the AI Dialogue can build upon, including the OECD AI Principles, UNESCO's Recommendation on the Ethics of AI, the G7 Hiroshima AI Process, as well as regional regulatory efforts such as the EU AI Act. In parallel, industry-led research and safety initiatives, including work by leading AI labs on alignment, safety, and human-AI interaction, are also contributing to the broader governance landscape. These initiatives have been essential in establishing core principles such as fairness, transparency, accountability, and human oversight. However, many of them were designed before the recent shift toward conversational and emotionally responsive AI systems. As a result, they do not fully capture how AI is now being used in practice, particularly in relational and emotional contexts. This creates a gap that is already being felt at the level of everyday life. Across countries, families and young people are engaging with AI systems for emotional support, advice, and interaction, often without clear guidance or tools to understand these experiences. Existing frameworks provide important foundations, but they do not yet offer practical ways to navigate these new forms of use. The added value of the AI Dialogue lies in its ability to connect these initiatives while also updating them. In particular, it can help translate high-level principles into more concrete and accessible guidance, including the development of ethical tools and frameworks that address continuous interaction, emotional influence, and long-term impact. This is especially important for supporting those who are currently navigating these technologies with the least support, including families, educators, and young users. By integrating perspectives from behavioral sciences, philosophy, education, and youth engagement, the Dialogue can help ensure that governance is not only coherent across institutions, but also meaningful in practice. In this sense, the AI Dialogue can act as a bridge between existing frameworks and emerging realities, helping governance evolve in step with how AI is actually being used.

Different stakeholders can contribute most meaningfully to the AI Dialogue if the format allows them not only to share expertise, but to engage with how AI systems are actually experienced in practice. Today, many discussions remain highly technical or policy-driven, which can limit the ability of policymakers to fully grasp how relational and immersive AI systems operate in real-life contexts. To address this gap, the Dialogue could include dedicated sessions where policymakers and regulators directly engage with these technologies, not only conceptually, but experientially. For example, interactive sessions could simulate how conversational AI is used for emotional support, decision-making, or relationship guidance, allowing participants to better understand the dynamics of trust, dependency, and influence. In addition, interdisciplinary immersive formats could be particularly valuable. Bringing together behavioral scientists, philosophers, designers, and technologists in guided sessions could help unpack how these systems shape perception, emotion, and behavior. Live reflection exercises or facilitated discussions could allow participants to experience, question, and critically assess these dynamics in real time. In my own work engaging with global experts across sectors, I have found that these more interactive and reflective formats often lead to deeper insights than traditional panel discussions, particularly when addressing complex and evolving forms of AI use. In sum, stakeholders should be encouraged to contribute not only positions, but also uncertainties, tensions, and open questions. AI governance is still emerging, and meaningful participation depends on creating spaces where learning, reflection, and dialogue can take place alongside policy development.

Several voices remain underrepresented in global discussions on AI governance, particularly those who focus on the human, relational, and existential dimensions of AI. Today, much of the debate is shaped by legal, technical, and economic perspectives. While these are essential, they do not fully address the deeper questions emerging from how AI is actually being used. Why are people increasingly turning to AI systems for emotional support rather than to other humans? Why do many families feel unequipped to talk about these technologies? Why are some individuals becoming dependent on AI even for simple tasks, such as writing a short message? And what does this shift reveal about our social bonds, our sense of autonomy, and our ways of relating to one another? These are not purely technical or regulatory questions. They are questions about human behavior, meaning, and vulnerability. They require the active involvement of disciplines such as philosophy, psychology, education, and the social sciences. In my work as a Resident Philosopher, engaging with researchers, practitioners, and communities across different regions, I have observed that these perspectives are often discussed informally, but remain insufficiently integrated into formal governance processes. As a result, there is a risk that policies are developed without fully understanding the underlying social dynamics they seek to address. To include these voices, the Dialogue should create dedicated spaces for interdisciplinary exchange, where philosophical and human-centered perspectives are treated as core inputs rather than complementary reflections. It should also integrate contributions from families, educators, and young people, whose experiences provide critical insight into how AI is shaping everyday life. Including these perspectives is essential not only for representation, but for developing governance approaches that are grounded in a realistic understanding of human experience in the age of AI.

To foster meaningful dialogue, the AI Dialogue should move beyond traditional panel formats and create spaces where participants can directly experience how AI systems interact with human emotions, cognition, and behavior. One powerful approach would be immersive sessions where policymakers, researchers, and other stakeholders engage with conversational AI systems in real time. For example, participants could interact with AI tools designed for emotional support, decision-making, or companionship, and then reflect collectively on how these interactions feel. Experiencing these systems firsthand can reveal dynamics of trust, validation, dependency, or persuasion that are difficult to fully grasp through abstract discussion alone. Another innovative format would involve interdisciplinary sessions focused on how the human brain responds to these technologies. Bringing together neuroscientists, behavioral scientists, philosophers, and designers could help unpack how memory, personalization, and emotional alignment shape user experience over time. For instance, sessions could explore how AI systems build a sense of continuity through memory, or how personalized responses can reinforce certain beliefs or emotional states. The Dialogue could also include guided reflection exercises, where participants are invited to examine their own reactions, expectations, and levels of reliance when interacting with AI. This creates space not only for technical analysis, but for deeper questioning about autonomy, agency, and human connection. In my experience working at the intersection of AI and human behavior, these experiential and reflective formats often generate insights that more traditional discussions do not surface. Finally, integrating youth and family perspectives into these interactive sessions can ensure that the Dialogue remains grounded in real-world experiences. Innovation in format is essential if we want governance conversations to keep pace with how these technologies are actually shaping human life.

Several existing initiatives provide important foundations for effective AI governance, particularly in establishing principles and accountability mechanisms. Frameworks such as the EU AI Act, the GDPR, the OECD AI Principles, and UNESCO's Recommendation on the Ethics of AI have been instrumental in shaping expectations around safety, transparency, and human oversight. At the same time, industry practices are beginning to evolve in response to more complex challenges. For example, leading AI companies are increasingly investing in interdisciplinary safety teams, including behavioral scientists, psychologists, and neuroscientists, to better understand how users interact with AI systems in real-world contexts. Research efforts, such as recent work on "emotional patterns" in AI systems, also reflect a growing awareness that AI does not only process information, but can shape interaction dynamics and user experience over time. However, these approaches remain largely focused on principles, system performance, or isolated risks. There is still a gap when it comes to governing AI systems that operate through continuous interaction, personalization, and emotional engagement. One promising direction is the development of more experiential and human-centered governance practices. This includes creating spaces where policymakers and stakeholders can directly engage with AI systems to better understand how they influence perception, trust, and decision-making. It also includes integrating insights from behavioral sciences and philosophy into governance processes, to address questions that are not purely technical, such as dependency, autonomy, and relational dynamics. In my experience working with interdisciplinary communities across sectors, these more reflective and practical approaches can help translate abstract principles into more meaningful and actionable forms of governance, especially for families, educators, and young people navigating these technologies in reality.