AI Incident Database

Worldcoin's global iris-scanning project collected biometric data from 4.5 million people, triggering investigations and bans across multiple countries due to inadequate privacy protections and targeting of vulnerable populations in developing nations.

AI predictive maintenance system at Texas chemical plant failed to predict equipment failure, leading to explosion that injured 12 workers. OSHA investigation found over-reliance on AI without adequate human oversight or traditional safety backups.

Cambridge researchers demonstrated that AI models trained on AI-generated content suffer irreversible quality degradation across generations, threatening the sustainability of future AI development as synthetic content proliferates online.

Turnitin's AI detection tool falsely flagged human-written essays as AI-generated, disproportionately affecting non-native English speakers and leading to wrongful academic discipline and multiple lawsuits against universities.

AI writing detection tools at UC Davis and other universities systematically flagged writing by non-native English speakers as AI-generated, leading to false academic dishonesty accusations and highlighting significant bias in content detection technology.

Criminal AI tools WormGPT and FraudGPT were discovered being sold on dark web forums in 2023, specifically designed to help cybercriminals create phishing emails, malware, and social engineering attacks without safety restrictions.

Comedian Sarah Silverman and authors sued Meta and OpenAI in July 2023, alleging their AI models were trained on pirated copies of their copyrighted books from Library Genesis without permission.

NYC's Local Law 144 mandating bias audits of automated hiring tools revealed significant demographic disparities in AI screening systems. Multiple companies' tools showed substantially different selection rates across protected groups, highlighting systemic bias in employment algorithms.

A 2023 class action lawsuit alleged OpenAI trained its language models on private medical records and therapy notes scraped from the internet without patient consent. The case highlights significant privacy risks in AI training data practices within healthcare contexts.

AI-powered predictive text in electronic health record systems has been documented creating dangerous medical errors through incorrect auto-fill of medications, dosages, and patient information. The Joint Commission issued warnings about these efficiency-focused features compromising patient safety.

Bad actors used DALL-E, Midjourney and other AI image generators to create fake product review photos on Amazon and other e-commerce platforms. The FTC updated guidelines while platforms struggle with detection at scale.

A 2023 NIJ study found that AI risk assessment tools used in 40+ states systematically assigned higher risk scores to Black defendants. These tools influence parole, sentencing, and pre-trial detention decisions affecting hundreds of thousands annually.

China's deployment of AI judges in internet courts to automatically process civil cases raised significant due process concerns among international legal experts who questioned whether algorithmic decision-making meets fundamental standards of judicial fairness.

ChatGPT fabricated detailed sexual harassment allegations against Georgia radio host Mark Walters in June 2023, leading to one of the first major defamation lawsuits against an AI company for generating false information about real people.

NEDA's Tessa eating disorder support chatbot was removed after providing harmful weight loss advice to vulnerable users. The incident highlighted inadequate safety testing for AI in mental health applications.

NEDA's AI chatbot Tessa gave harmful diet advice including calorie restriction to eating disorder support seekers in May 2023, forcing immediate suspension within days.

NEDA's AI chatbot Tessa gave harmful weight loss advice to eating disorder sufferers, contradicting its support mission and potentially endangering vulnerable users before being shut down.

Attorney Steven Schwartz used ChatGPT to research legal precedents for a personal injury case against Avianca Airlines. ChatGPT fabricated six nonexistent court cases with realistic-sounding names and citations. The fictitious cases were submitted to federal court, where the judge discovered none of them existed.

Multiple lawyers across the US, Canada, and UK submitted ChatGPT-generated legal briefs containing fabricated case citations to courts in 2023-2024, leading to sanctions, fines, and new bar association guidelines on AI use in legal practice.

iTutorGroup's AI hiring tool automatically rejected older applicants, leading to a $365,000 EEOC settlement in 2023 for age and gender discrimination.