AI Incident Database

Yelp's AI review filtering system allegedly suppressed legitimate positive reviews for small businesses while preserving negative ones, creating financial pressure to buy ads. Class action lawsuits were filed but courts ruled Yelp had no obligation to display reviews fairly.

Apple Maps' navigation AI directed drivers into dangerous situations including airport runways and nonexistent roads due to faulty routing algorithms and incomplete map data, affecting hundreds of thousands of users worldwide.

Knight Capital's trading algorithm malfunctioned on August 1, 2012, executing millions of erroneous stock trades that lost $440 million in 45 minutes, nearly bankrupting the firm and forcing its acquisition by Getco.

JPMorgan's flawed VaR model change in 2012 systematically underestimated trading risk, failing to flag the London Whale's massive positions and contributing to $6.2 billion in losses. The incident resulted in over $920 million in regulatory fines.

MIT researcher Les Perelman demonstrated that ETS's e-rater automated essay scoring system used for GRE and GMAT could be gamed with verbose nonsense while showing bias against non-standard English dialects.

Target's pregnancy prediction algorithm correctly identified a teenage customer's pregnancy through purchase patterns and sent baby product coupons to her home, inadvertently revealing her condition to her father before she had disclosed it.

The Fukushima nuclear disaster revealed critical flaws in automated safety systems that failed to handle cascading beyond-design-basis events, leading to three reactor meltdowns and massive environmental contamination.

On May 6, 2010, algorithmic trading systems caused the Dow Jones to plunge 998 points in minutes, temporarily erasing $1 trillion in market value before recovering. The incident exposed critical vulnerabilities in automated high-frequency trading systems.

Indiana's automated welfare eligibility system wrongly denied benefits to over one million residents due to rigid programming that couldn't handle minor procedural issues. The state ultimately paid $1.37 billion to settle lawsuits and terminated the system.

Automated welfare eligibility systems across multiple US states wrongfully terminated benefits for over one million vulnerable Americans between 2007-2020. The algorithms contained systematic biases leading to $1.2 billion in estimated harm through wrongful benefit cuts.

The UK Post Office Horizon IT system generated false accounting discrepancies from 1999-2015, leading to over 900 wrongful prosecutions of sub-postmasters for theft and fraud, making it one of the largest miscarriages of justice in British history.

Faulty Fujitsu Horizon IT system caused false accounting shortfalls leading to wrongful prosecution of over 900 UK sub-postmasters between 1999-2015. Described as Britain's largest miscarriage of justice, with ongoing public inquiry and compensation efforts.