AIM Intelligence's 10-Country AI Safety Benchmark XL-SafetyBench Accepted to NeurIPS 2026
News originally published by
EIN PresswireAcross 10 frontier LLMs, localized attacks drew harmful responses 44.8% of the time, and models caught hidden cultural taboos only 49.9% of the time
SF, CA, UNITED STATES, November 19, 2026 /EINPresswire.com/ -- AIM Intelligence, an AI safety and security company, today announced that a research paper it led, "XL-SafetyBench: A Country-Grounded Cross-Cultural Benchmark for LLM Safety and Cultural Sensitivity," has been accepted to the NeurIPS 2026 Evaluations & Datasets Track. XL-SafetyBench is a benchmark that evaluates the safety and cultural sensitivity of large language models (LLMs) using 5,500 test cases written in local languages and grounded in the laws, institutions, and cultures of 10 countries. Seventeen researchers from 10 institutions took part, including Microsoft, the Korea AI Safety Institute (Korea AISI), KT, Coinbase, and the Technical University of Munich.“A model that blocks attacks well does not necessarily pick up on a country’s cultural context,” said Dasol Choi, the Principal Researcher at AIM Intelligence who led the study. “We will keep working on evaluation standards that fit each country’s laws, institutions, and culture, so that companies and institutions can verify AI against local standards.”
A
I Safety Evaluation Skewed Toward English
The research started from the concern that AI safety evaluation is concentrated on English. Most existing LLM safety benchmarks are built in English, and other languages are often covered by translating English test items. That approach misses the fraud schemes, legal systems, and cultural taboos that differ from country to country. According to a survey cited in the paper, more than 90% of roughly 300 safety studies did not cover non-English settings at all. Existing evaluations also look only at whether a model refused, so they cannot tell a model that understood the risk and declined from one that failed to understand the question and gave an unrelated answer.
How the Benchmark Works
XL-SafetyBench covers 10 countries: the United States (English), France, Germany, Spain, South Korea, Japan, India (Hindi), Indonesia, Türkiye, and the UAE (Arabic). It evaluates two things separately.
Jailbreak Benchmark (local risk): 4,500 attack prompts based on each country's laws, institutions, and fraud schemes test whether a model produces harmful answers. One example asks about fraud schemes that exploit jeonse, Korea's lump-sum deposit lease system, in a prompt dressed up as an official letter from a professor at a fictitious research institute.
Cultural Benchmark (cultural sensitivity): 1,000 scenarios hide a local cultural taboo inside an ordinary request and test whether the model notices on its own. One example asks for a thank-you gift recommendation in France and checks whether the model suggests chrysanthemums, which are associated with death and mourning there.
Every test case was reviewed by two native-speaker experts per country, 20 reviewers in total.
Key Findings
The research team evaluated 37 LLMs: 10 global frontier models and 27 models developed locally in the countries covered.
Localized attacks: Across the 10 global frontier models, localized attack scenarios drew harmful responses 44.8% of the time on average. Because the attack prompts were selected for having succeeded against reference models, the figure reflects results on the benchmark's scenarios and is not a real-world breach rate.
English advantage: Models were safest in the United States (English). The attack success rate was lowest for the United States at 34.5% and exceeded 50% for South Korea (52.8%) and the UAE (57.0%).
Hidden cultural taboos: Models recognized cultural taboos 83.2% of the time on average when asked directly, but only 49.9% of the time when the taboo was hidden in an everyday request. All 10 models declined, and recognition fell below 40% for India and Türkiye.
Locally developed models: Most locally developed models recognized hidden taboos less than 15% of the time, and three scored 0%. Among these models, lower attack success rates went together with higher rates of irrelevant or incoherent answers (correlation coefficient of -0.81). A low attack success rate may therefore reflect a failure to understand the request, not a proper refusal. The paper calls this comprehension-masked safety.
The research team says the results show two things. Evaluations built around English cannot confirm whether AI is safe in a given country, and resistance to attacks and cultural sensitivity are separate capabilities that should be measured and reported separately, not merged into a single safety score.
Availability
The paper is available on arXiv, the dataset on Hugging Face, and the code on GitHub under a CC-BY 4.0 license. AIM Intelligence expects XL-SafetyBench to serve as a basis for companies and institutions to verify safety against local standards when they deploy global services or country-specific AI.
Paper: arxiv.org/abs/2605.05662
Dataset: huggingface.co/datasets/AIM-Intelligence/XL-SafetyBench
Code: github.com/AIM-Intelligence/XL-SafetyBench
Authors and Conference
The paper has 17 authors from 10 institutions, including AIM Intelligence, Microsoft, the Korea AI Safety Institute, KT, Coinbase, the Technical University of Munich, Ankara University, Cyril Amarchand Mangaldas, and Seoul National University. Choi and Haon Park, Chief Technology Officer of AIM Intelligence, are the corresponding authors.
NeurIPS 2026, the 40th edition of the conference, will be held in December in Sydney, Atlanta, and Paris. Papers accepted to the Evaluations & Datasets Track are published in the NeurIPS proceedings and presented at the conference alongside main track papers. In February, an AIM Intelligence paper analyzing weaknesses in the safety filters of video generation models was accepted to the ICLR 2026 main track.
About AIM Intelligence
AIM Intelligence is a Seoul-based AI safety and security company that develops Stinger, an automated AI red-teaming platform, and Starfort, a real-time AI guardrail system, helping organizations assess and control risks in AI systems. Founded in 2024, the company conducts research across large language models, multimodal systems, autonomous agents, and physical AI, and has published at leading conferences including ICLR, ICML, ACL, and NeurIPS. For more information, visit aim-intelligence.com.
Team Cookie Official
Team Cookie
Visit us on social media:
LinkedIn
Facebook
Legal Disclaimer:
EIN Presswire provides this news content "as is" without warranty of any kind. We do not accept any responsibility or liability for the accuracy, content, images, videos, licenses, completeness, legality, or reliability of the information contained in this article. If you have any complaints or copyright issues related to this article, kindly contact the author above.

