Oxford / The Alan Turing Institute

AI Safety must get serious about preventing terrorist misuse. In the past year alone, there have been 16 AI-enabled terror attacks and foiled plots in Canada, the US, Finland, and Indonesia. Moreover, 8 in 10 commercial chatbots have insufficient safeguards to prevent terrorist misuse, according to a new study from CCDH and CNN's Investigative Unit. AI Safety teams and funding bodies have always focused on long-term, potential risks while largely ignoring near term harms. But as models capabilities reach the stage of being directly useful for attack planning, target selection, weapons procurement, chatbot radicalisation at scale, and the industrial production of terrorist propaganda (TVEC) in all multi-modal formats (image, video, audio, text, code). There are both technical solutions and governance frameworks available that could prevent many if not all of these attacks, but we still do not see meaningful engagement from AI labs or the existential risk/EA community. My greatest fear is a mass casualty attack that employed AI to help bring it about. This doesn't necessarily mean the most advanced, sophisticated CBRN risks that existential risk researchers look at, it is often much less sophisticated misuse cases that cause the most real-world violence. If we care about preventing more AI enabled attacks we must engage the researchers working on this challenge and implement pragmatic & actionable solutions. We dont have to chose between long-term existential risk and near-team risk. We can & must do both, at the same time. If we can walk & chew gum, then we can tackle both simultaneously. Both are critical for making AI safe, reliable, and aligned.

• Safe, secure and trustworthy AI
• Open-source software, open data and open AI models
• Social, economic, ethical, cultural, linguistic and technical implications of AI
• Protection and promotion of human rights

I would have selected all of them, as they bleed over into the others (ie., governance approaches and protection & promotion of human rights are inextricably linked.

AI Safety must get serious about preventing terrorist misuse. In the past year alone, there have been 16 AI-enabled terror attacks and foiled plots in Canada, the US, Finland, and Indonesia. Moreover, 8 in 10 commercial chatbots have insufficient safeguards to prevent terrorist misuse, according to a new study from CCDH and CNN's Investigative Unit. AI Safety teams and funding bodies have always focused on long-term, potential risks while largely ignoring near term harms. But as models capabilities reach the stage of being directly useful for attack planning, target selection, weapons procurement, chatbot radicalisation at scale, and the industrial production of terrorist propaganda (TVEC) in all multi-modal formats (image, video, audio, text, code). There are both technical solutions and governance frameworks available that could prevent many if not all of these attacks, but we still do not see meaningful engagement from AI labs or the existential risk/EA community. My greatest fear is a mass casualty attack that employed AI to help bring it about. This doesn't necessarily mean the most advanced, sophisticated CBRN risks that existential risk researchers look at, it is often much less sophisticated misuse cases that cause the most real-world violence. If we care about preventing more AI enabled attacks we must engage the researchers working on this challenge and implement pragmatic & actionable solutions. We dont have to chose between long-term existential risk and near-team risk. We can & must do both, at the same time. If we can walk & chew gum, then we can tackle both simultaneously. Both are critical for making AI safe, reliable, and aligned.

Low-resource languages are being overlooked in red-teaming and evals. Focus on long term existential risk instead of near-term harms which are already resulting in real-world violence. Lack of subject matter expertise in AI labs from the counterterrorism and intelligence communities.

It has enormous potential to convene the right leaders and begin pushing us in a more holistic direction.

More engagement with Chinese AI Labs and safety institutes is critical to ensure terrorists, criminals, and malicious actors cannot bypass our safeguards in the US and Europe by simply using a Chinese model (especially China's open-source models which now lead rankings on Hugging Face)

Further engagement with Chinese AI Labs and safety institutes is critical to ensure terrorists, criminals, and malicious actors cannot bypass our safeguards in the US and Europe by simply using a Chinese model (especially China's open-source models which now lead rankings on Hugging Face)

Same as above. Further engagement with Chinese AI Labs and safety institutes is critical to ensure terrorists, criminals, and malicious actors cannot bypass our safeguards in the US and Europe by simply using a Chinese model (especially China's open-source models which now lead rankings on Hugging Face). We also need to invest more in low-resource languages from around the world both during training and testing.

To reach many of the stakeholders and meaningfully engage them, the work of the Safe AI Forum and Concordia is critical. They focus on engaging Chinese stakeholders, and it would be very effective to partner with them

Our recent AISI-Turing report lays out 15 policy concrete & actionable recommendations to prevent AI misuse by adversarial actors ranging from terrorist to criminals to hostile foreign states. Our recommendations are scoped specifically for government, industry, and civil society Full details & recommendations https://cetas.turing.ac.uk/publications/adding-fuel-to-fire