Skip to content

Group for the Advancement of Psychiatry; Boston University Chobanian & Avedisian School of Medicine

Academia Western Europe and Other States

Responses

In your opinion, what outcomes would make the first Global Dialogue on AI Governance a success?

The first Global Dialogue would be successful if it produces three concrete outcomes. First, it should establish a shared evidence agenda that connects the Independent International Scientific Panel's assessments to practical governance questions faced by countries, health systems, clinicians, patients, and communities. Second, it should identify minimum safeguards for high-impact AI settings, including transparent scope-of-action labels, meaningful human oversight, accountability and redress pathways, consent processes, subgroup equity monitoring, incident reporting, and rollback triggers. Third, it should create a recurring mechanism for stakeholder evidence, including lived-experience and frontline case studies from domains where AI affects safety, dignity, and access. In healthcare, success should include explicit recognition that high-stakes settings such as geriatric psychiatry, dementia care, long-term care, and discharge planning can serve as sentinel environments for testing whether AI governance protects people under conditions of vulnerability, uncertainty, and dependency. The Dialogue should leave participants with a clear pathway from principles to implementation: what evidence should be collected, who should be accountable, how harms will be detected, and how effective safeguards will be shared across countries and sectors.

From your perspective, which of the following thematic areas identified by the General Assembly Resolution 79/325 for the AI Dialogue reflect your priorities for urgent action and active engagement?

  • Safe, secure and trustworthy AI
  • Social, economic, ethical, cultural, linguistic and technical implications of AI
  • Protection and promotion of human rights
  • Transparency, accountability, and human oversight

Please briefly explain your selection.

4

These priorities are linked in high-stakes health and social-care AI. Safety and trustworthiness cannot be assessed only by model accuracy; they must include what patients, caregivers, and clinicians experience. In geriatric psychiatry, AI may affect older adults with cognitive impairment, sensory loss, grief, frailty, multiple illnesses, polypharmacy, and dependence on caregivers or institutions. These conditions raise human-rights concerns around dignity, autonomy, informed consent, privacy, non-discrimination, and access to care. Transparency, accountability, and human oversight are therefore essential. A system that cannot explain its role, show uncertainty, allow opt-out or appeal, document escalation, and monitor equity should not be scaled in high-impact settings. The ethical, cultural, linguistic, and technical dimensions are inseparable: interfaces must accommodate disability, language, culture, and cognitive variability. The submission proposes Humane Intelligence and the Moral Grid Operational Index (MGOI) as a practical, clinician-centered way to translate these priorities into auditable evidence for pilots, procurement, monitoring, and policy learning.

In your opinion, are there any cross-cutting or emerging issues not captured by the listed themes above? If so, please explain.

4

A cross-cutting issue that deserves explicit attention is relational safety: whether AI systems preserve dignity, trust, autonomy, and meaningful human connection in settings where people are vulnerable or dependent. Existing governance often focuses on algorithms, infrastructure, and formal transparency. In health and social care, however, harms can arise through subtle relational failures: simulated empathy without accountability, rushed or confusing consent, decontextualized recommendations, automation bias, surveillance without meaningful benefit, or tools that quietly shift burdens onto patients, caregivers, or clinicians. Another emerging issue is long-conversation safety. AI systems used for companionship, behavioral support, mental health, coaching, or patient portals may interact with people across extended, emotionally charged exchanges. Short-turn testing may miss cumulative drift, over-agreement, boundary erosion, escalation failures, or progressive loss of context. Global AI governance should therefore include evaluation of relational performance, consent integrity, uncertainty communication, escalation behavior, and sustained-interaction safety, not only one-time accuracy or compliance.

How are the governance gaps and related developments/advances in the thematic areas you selected above affecting your country, region, or sector? Please highlight the most significant challenges.

In healthcare and aging care, AI is entering documentation, triage, discharge planning, remote monitoring, patient portals, companionship tools, medication support, prior authorization, scheduling, and population management. Governance gaps are most visible where tools affect vulnerable people but are not treated as clinically consequential because they are labeled administrative, supportive, or low risk. For older adults, risks include cognitive impairment, sensory loss, digital exclusion, social isolation, grief, frailty, and dependence on caregivers or institutions. AI can improve access, reduce clinician burden, support loneliness and sleep interventions, identify caregiver strain, and make information clearer. But without robust safeguards, it can also produce premature clinical closure, biased access decisions, inappropriate medication guidance, dehumanizing surveillance, opaque discharge scores, or erosion of consent. The opportunity is to use geriatric psychiatry and long-term care as proving grounds for humane, risk-aware AI governance. These settings concentrate complexity, vulnerability, and high consequence. If AI can be made safe, transparent, and relationally trustworthy here, the lessons can generalize to other high-impact domains.

What role can the AI Dialogue play in advancing international cooperation on AI governance?

The AI Dialogue can serve as a boundary-spanning forum that turns scientific assessment into practical cooperation. It should connect the Independent International Scientific Panel's evidence syntheses with real-world governance questions: what must be measured, what safeguards are proportionate to risk, how should incidents be reported, and how should accountability work across vendors, deployers, clinicians, public agencies, and affected communities. The Dialogue can advance cooperation by developing interoperable evidence schemas for high-impact AI deployments. These should be flexible enough for countries with different resources, yet specific enough to support comparison. For healthcare AI, a common schema could include use-case labels, scope-of-action limits, safety testing, consent process, human escalation, subgroup performance, adverse events, overrides, accountability nodes, and rollback triggers. The Dialogue should also convene sentinel-domain case studies from sectors such as health, labor, education, public benefits, and justice. This would support concrete learning, not only abstract principle-sharing, while preserving room for local adaptation.

What are some of the existing initiatives, partnerships, or mechanisms that the AI Dialogue should build upon or connect with, and what added value could the AI Dialogue bring?

The Dialogue should build on existing international and sectoral governance work rather than duplicate it. Relevant foundations include World Health Organization guidance on ethics and governance of AI for health, NIST's AI Risk Management Framework, the JAMA Summit's healthcare AI lifecycle priorities, ONC algorithm transparency and decision-support-intervention requirements, FDA guidance on predetermined change control plans for AI-enabled device software, HL7 consent infrastructure, and emerging certification and post-deployment monitoring models for health AI. In Europe and other regions, high-risk AI frameworks also provide useful documentation, oversight, and rights-protection concepts. The added value of the Dialogue would be to connect these efforts across sectors and countries, identify gaps, and promote interoperable minimum evidence standards. The Dialogue should also welcome operational tools from frontline domains, including clinician-centered scorecards and implementation protocols. Humane Intelligence and MGOI are examples of such tools: they translate ethical principles into observable behaviors, auditable evidence, risk-tiered oversight, and monitoring routines.

How can different stakeholders contribute to the AI Dialogue? Please share recommendations for the format and structure of the AI Dialogue.

  • Different stakeholders should contribute different forms of evidence. Patients, older adults, caregivers, disability advocates, and community organizations can describe lived experience, comprehension barriers, consent concerns, and trust impacts. Clinicians and frontline workers can identify workflow risks, safety incidents, automation bias, and the difference between useful support and harmful substitution. Developers and vendors can provide model cards, data lineage, safety testing, known limitations, incident logs, and change-control plans. Regulators, standards bodies, and researchers can help evaluate evidence quality, comparability, and generalizability. The Dialogue format should include: (1) evidence brief submissions with structured templates
  • (2) high-stakes case-study sessions
  • (3) metric labs to test what can be measured across settings
  • (4) accessibility-focused consultations in multiple languages and formats
  • and (5) public synthesis documents that explain how stakeholder inputs changed the agenda. For health AI, a dedicated track on vulnerable populations and clinical/social-care deployment would help ensure that policy discussions address safety, dignity, consent, equity, and accountable implementation.

Which voices, communities, or perspectives are currently underrepresented in global discussions on AI governance? How could they be included?

  • Several groups are underrepresented in global AI governance despite being highly affected by AI deployment. In health and social care, these include older adults with cognitive impairment, sensory impairment, frailty, serious mental illness, or limited digital access
  • residents of long-term care and assisted living facilities
  • caregivers and family decision-makers
  • adults whose decisions are mediated by health care proxies, guardians, or institutional rules
  • people with disabilities
  • rural and low-resource communities
  • multilingual and minoritized communities
  • and people whose data are used but who do not experience clear benefit. Frontline clinicians, nurses, social workers, direct-care staff, and patient-safety leaders are also underrepresented compared with technology developers and institutional decision-makers. They see where AI actually changes care. Inclusion requires more than open invitations. The Dialogue should provide accessible, multilingual, hybrid participation
  • plain-language materials
  • compensation or logistical support for lived-experience contributors
  • and structured pathways for clinicians and caregivers to contribute case evidence without violating privacy.

What innovative engagement formats could most effectively foster meaningful and dynamic engagement during the AI Dialogue?

The Dialogue could use sentinel-domain studios: structured sessions where a concrete AI use case is examined from the perspectives of affected people, frontline implementers, developers, ethicists, and policymakers. For example, a geriatric psychiatry studio could examine an AI companion, discharge-readiness score, ambient documentation tool, or patient-portal chatbot and ask: What is the scope of action? What could go wrong? How is consent obtained and renewed? What uncertainty is disclosed? Who can stop or appeal the output? Which subgroups might be harmed? What evidence triggers rollback? Other useful formats include anonymized incident-review sessions, red-team clinics focused on vulnerable populations, scorecard field tests, Delphi-style consensus exercises, and metric labs that compare proposed indicators across settings. Asynchronous formats should allow people with caregiving responsibilities, disabilities, or limited connectivity to participate meaningfully. The outputs should be transparent: summaries, evidence maps, unresolved questions, and a record of how input shaped recommendations.

Please share examples of policies, practices, platforms, or approaches that promote effective AI governance or offer concrete solutions to addressing its challenges.

6

A practical example is the Humane Intelligence and Moral Grid Operational Index (MGOI) approach for health and social care AI (https://doi.org/10.1016/j.jagp.2025.12.012). Humane Intelligence is organized around four pillars: Relational Intelligence, Transparency with Care, Reciprocity and Consent, and Ethical Governance in Strategic Regions. MGOI operationalizes those pillars through a decision grid, 0-2 scorecard anchors, a minimum evidence set, red-flag gates, risk tiers, and monitoring routines. Related practices include: scope-of-action labels; hard no-advice zones for medication dose changes, crisis situations, involuntary care, and eligibility denial; human-in-the-loop requirements for high-stakes decisions; consent receipts and consent renewal; uncertainty indicators; subgroup equity monitoring with rollback triggers; change logs for model updates; incident reporting; stop/undo/explain/redress pathways; and procurement checklists that define criteria, accountability, support, and recourse. A proposed supplement, REACH-LC, focuses on long-conversation safety by testing extended interactions for fabrication, omission, decontextualization, overreach, calibration error, and agreeableness bias. These approaches make AI governance observable, auditable, and usable by clinicians and communities. I can submit an evidence-linked policy brief in my personal capacity to support the UN AI Scientific Panel and Global Dialogue on AI Governance, upon request.