AI Incident Database

IBM Watson for Oncology recommended dangerous cancer treatments, including chemotherapy for patients with severe bleeding, due to flawed training on hypothetical cases rather than real outcomes data.

South Wales Police facial recognition system at 2018 Champions League final falsely identified 2,470 innocent people with 92% false positive rate. Court later ruled the deployment unlawful, violating human rights.

Amazon Alexa recorded a Portland couple's private conversation about hardwood floors and sent the audio file to a random business contact due to a series of voice recognition errors and lack of user confirmation for sensitive actions.

Google Duplex demonstrated realistic phone conversations with businesses without disclosing its AI nature, sparking widespread ethical concerns about AI deception and transparency requirements.

Google's 2018 Duplex demo showed AI making restaurant reservations without identifying itself as non-human, sparking widespread ethical concerns about AI deception and leading to mandatory disclosure requirements.

Tesla Autopilot failed to detect a concrete barrier on Highway 101, killing Apple engineer Walter Huang in March 2018. NTSB found the system misclassified the barrier and accelerated into it.

An Uber self-driving test vehicle struck and killed 49-year-old pedestrian Elaine Herzberg in Tempe, Arizona, marking the first documented fatality involving a fully autonomous vehicle. The car's AI perception system detected Herzberg but repeatedly misclassified her and failed to initiate emergency braking.

Uber's self-driving test vehicle fatally struck pedestrian Elaine Herzberg in March 2018 when AI perception systems misclassified her multiple times and failed to predict her path. NTSB investigation revealed disabled emergency braking and inadequate safety protocols.

Cambridge Analytica used AI to analyze Facebook data from 87 million users, building psychographic profiles for targeted political manipulation in the 2016 election and Brexit. The incident resulted in a $5B FTC fine and raised critical questions about AI's role in democratic processes.

Grammarly's browser extension contained a critical vulnerability that exposed all 22 million users' documents to any website they visited, discovered by Google Project Zero researcher Tavis Ormandy.

Dartmouth researchers found that COMPAS recidivism prediction AI used in criminal sentencing performed no better than random untrained people, despite being widely deployed across US courts.

A Navya autonomous shuttle in Las Vegas froze when a truck backed into it on the first day of public trials, highlighting limitations in autonomous vehicle evasive action programming despite correct threat detection.

The cargo ship El Faro sank in Hurricane Joaquin after AI weather routing systems directed it into the storm's path, killing all 33 crew members. The NTSB investigation revealed the automated systems prioritized efficiency over safety.

Facebook's AI mistranslated a Palestinian man's Arabic 'Good morning' post as 'Attack them' in Hebrew, leading to his arrest by Israeli police before the error was discovered.

Australia's Robodebt scheme used flawed automated income averaging to generate over 400,000 false welfare debt notices from 2016-2019. The Royal Commission found the system illegal, leading to $1.8 billion in settlements and major government accountability reforms.

Microsoft's Zo chatbot, launched in 2017 as an improved successor to Tay, still produced offensive content including religious bias when users found ways to bypass its safety filters.

YouTube Kids' recommendation algorithm promoted disturbing content disguised as children's programming to millions of children. The FTC fined YouTube $170 million in 2019 for COPPA violations related to this incident.

Palantir's AI system enabled ICE to analyze data from schools and social services to identify and target undocumented immigrants for deportation, raising significant civil liberties concerns.

Consumer Reports and ProPublica investigations revealed that State Farm, Allstate, and other major insurers used AI pricing models that systematically charged higher premiums in minority neighborhoods, affecting millions of consumers despite controlling for legitimate risk factors.

Waze's AI routing algorithm directed heavy traffic through residential neighborhoods and school zones, creating safety hazards and prompting cities to implement countermeasures against algorithmic routing decisions.