AI Incident Database

The EU-funded iBorderCtrl AI lie detector was deployed at borders in Hungary, Latvia, and Greece despite lacking scientific validation for micro-expression deception detection.

Optum's widely-used healthcare risk prediction algorithm showed severe racial bias, requiring Black patients to be significantly sicker than white patients to receive the same care recommendations, affecting an estimated 200 million patients nationwide.

Optum's widely-used healthcare risk algorithm systematically underestimated care needs for Black patients by using healthcare spending as a proxy for health status, affecting over 10 million patients across major health systems.

Babylon Health's AI triage system in the NHS GP at Hand service incorrectly classified serious medical conditions as non-urgent. BBC investigation revealed systematic failures that could have delayed critical care for patients.

LAPD's use of PredPol predictive policing software from 2011-2019 created feedback loops that disproportionately targeted Black and Latino neighborhoods, with multiple academic studies documenting systematic bias before the department ended the program.

Criminals used AI voice cloning technology to impersonate a parent company CEO, successfully deceiving a UK energy firm executive into authorizing a $243,000 wire transfer in one of the first documented deepfake fraud cases.

Academic research revealed Uber's AI surge pricing algorithm consistently charged higher prices in minority and lower-income neighborhoods due to supply-demand patterns that correlated with demographics.

Hong Kong authorities deployed AI facial recognition through smart lampposts and CCTV networks to identify pro-democracy protesters in 2019-2020, leading to arrests and systematic suppression of assembly rights.

Facial recognition cameras at London's King's Cross development operated without public knowledge for 18 months, processing millions of people's biometric data in violation of GDPR before being discovered and shut down.

CBP's facial recognition systems at US border crossings demonstrated significant bias, with higher error rates for people of color, women, and elderly travelers. GAO investigations revealed systematic disparities affecting millions of border crossers annually.

Apple's Siri assistant inadvertently recorded private conversations due to false wake word triggers, with contractors regularly hearing confidential medical information and intimate moments, leading to a $95M settlement.

DoorDash's AI payment algorithm systematically used customer tips to reduce driver base pay rather than supplement it, affecting approximately 250,000 drivers over two years before being exposed in 2019, resulting in a $2.5M FTC settlement.

Epic Systems' sepsis prediction AI tool deployed at University of Michigan showed a 67% false alert rate, with only 7% of predictions confirmed as sepsis, causing alert fatigue among clinicians.

LinkedIn's AI-powered job and skill recommendations showed systematic gender bias, suggesting administrative roles to women and executive positions to men. The company acknowledged the issue and implemented changes to reduce bias in their algorithms.

An autonomous electric bus struck a pedestrian in Vienna during 2019 trial operations when its AI sensor systems failed to detect the person crossing the street.

China deployed comprehensive AI surveillance systems including facial recognition, predictive policing, and data integration platforms to systematically identify and detain over one million Uyghur Muslims and other ethnic minorities in Xinjiang since 2017.

Chinese authorities deployed AI-powered surveillance systems from companies including Huawei and Hikvision to systematically track and profile Uyghur Muslims in Xinjiang, contributing to mass detention and persecution. The technology used facial recognition and behavioral analysis to automatically target individuals based on ethnicity.

Amazon's AI-powered warehouse productivity tracking system automatically terminated workers who couldn't meet algorithm-set quotas, leading to unsafe working speeds and injury rates significantly higher than industry averages.

Research revealed AI essay grading systems like e-rater systematically gave lower scores to non-native English speakers despite equivalent content quality, affecting standardized test outcomes for international students.

Bloomberg revealed Amazon employed thousands of contractors worldwide to listen to Alexa recordings from users' homes for speech training, exposing private conversations without adequate user consent and leading to privacy lawsuits.