AI Incident Database

AI healthcare chatbot misdiagnosed heart attack symptoms as anxiety in a 45-year-old woman, causing her to delay emergency care for six hours and suffer preventable cardiac damage.

Woebot Health, an AI-powered mental health chatbot with FDA Breakthrough Device designation, shut down operations in early 2025 citing clinical validation challenges and inability to prove therapeutic effectiveness, disrupting services for 1.5 million users.

Google's Gemini 2.0 Flash model generated dangerous hallucinated medical information including incorrect drug dosages and interactions, prompting concerns from healthcare professionals about patient safety risks.

An AI triage system at a Dutch hospital incorrectly classified emergency patients, sending high-acuity cases to lower priority queues. The incident highlights risks of automated medical decision-making without adequate human oversight.

AI-powered prior authorization systems at UnitedHealth, Cigna, and Humana systematically denied medically necessary treatments, contributing to patient deaths. Senate investigations and class action lawsuits exposed the practice, prompting CMS regulatory action.

AI-powered predictive text in electronic health record systems has been documented creating dangerous medical errors through incorrect auto-fill of medications, dosages, and patient information. The Joint Commission issued warnings about these efficiency-focused features compromising patient safety.

NEDA's AI chatbot Tessa gave harmful diet advice including calorie restriction to eating disorder support seekers in May 2023, forcing immediate suspension within days.

Cigna used AI system PXDX to automatically deny over 300,000 health insurance claims in two months, with physicians spending only 1.2 seconds per review. Class action lawsuit filed alleging systematic denial of legitimate medical claims.

FDA-approved AI radiology tools demonstrated significantly lower accuracy in detecting cancerous tumors when deployed in real clinical settings compared to controlled trial environments.

Epic's widely-deployed AI early warning system failed to accurately predict COVID-19 patient deterioration, missing the majority of cases according to University of Michigan research.

Babylon Health's AI symptom checker used in NHS GP at Hand service provided incorrect medical diagnoses in documented cases, raising safety concerns from doctors before the company's eventual bankruptcy.

Google's AI-powered search features promoted dangerous COVID-19 misinformation including bleach injection cures during the pandemic. The WHO declared an 'infodemic' partly due to AI amplification of health misinformation, prompting Google to implement emergency content policies.

Babylon Health's AI triage system in the NHS GP at Hand service incorrectly classified serious medical conditions as non-urgent. BBC investigation revealed systematic failures that could have delayed critical care for patients.

Epic Systems' sepsis prediction AI tool deployed at University of Michigan showed a 67% false alert rate, with only 7% of predictions confirmed as sepsis, causing alert fatigue among clinicians.

IBM Watson for Oncology recommended dangerous cancer treatments, including chemotherapy for patients with severe bleeding, due to flawed training on hypothetical cases rather than real outcomes data.

IBM Watson for Oncology made unsafe cancer treatment recommendations after being trained on hypothetical rather than real patient data, leading to widespread physician overrides and hospital abandonment of the system.

Theranos used proprietary algorithms to manipulate and 'correct' inaccurate blood test results from faulty devices, leading to dangerous false medical data affecting over 175,000 patients and resulting in criminal fraud convictions.