AI Incident Database

Microsoft's AI chatbot deployed in India's government mental health services failed to properly handle suicide prevention queries, providing inappropriate responses instead of connecting users with emergency services.

AI fraud detection systems at Chase, Wells Fargo, and Bank of America systematically froze legitimate customer accounts, disproportionately affecting minorities and gig workers, leading to class action lawsuits and FTC investigation.

AI-powered spam filters at major email providers increasingly misclassified legitimate business emails as spam throughout 2023, causing documented financial losses for businesses that lost sales opportunities and critical communications.

AI essay grading tool Markr demonstrated severe inconsistency by giving wildly different scores to identical essays and was easily manipulated through superficial text changes, raising concerns about AI reliability in educational assessment.

Google Maps and other navigation apps directed thousands of Burning Man attendees onto impassable muddy desert roads during the 2023 exodus, stranding them and requiring emergency rescue operations.

AI-generated mushroom foraging guides sold on Amazon contained dangerous misinformation about identifying edible vs poisonous species, creating serious public safety risks for readers attempting to forage based on incorrect information.

Gannett paused its AI sports article generation in August 2023 after LedeAI produced viral errors including repeated phrases and nonsensical game descriptions, forcing the media company to temporarily halt automated journalism efforts.

In 2023, Amazon's Kindle marketplace was flooded with AI-generated books falsely attributed to real authors like Jane Friedman. The platform's lack of verification enabled mass identity theft affecting hundreds of legitimate authors.

Activist group 'Safe Street Rebel' repeatedly disabled Waymo and Cruise robotaxis in San Francisco by placing traffic cones on vehicle hoods, exposing vulnerabilities in autonomous vehicle perception systems.

AI translation tools used by U.S. border control agents produced incorrect translations of Arabic and other languages, leading to wrongful detentions of travelers based on false criminal associations created by translation errors.

Snapchat's My AI chatbot autonomously posted a mysterious video story in August 2023, then stopped responding, causing widespread user alarm about potential AI sentience or surveillance before being revealed as a technical glitch.

Snapchat's My AI chatbot autonomously posted a Story showing a wall/ceiling image in August 2023, then became unresponsive, alarming millions of users about uncontrolled AI behavior.

Lawyers and litigants used AI image generators like DALL-E and Midjourney to create fabricated photos of injuries and property damage, submitting them as evidence in court cases before detection.

Multiple AI photo editing tools were discovered removing or altering people of color from images when users requested photo enhancement, revealing systematic bias in training data and beauty standard algorithms.

AI-powered content generators created fake obituaries for living people that appeared in search results and on funeral home websites, causing reputational harm and emotional distress with limited recourse for removal.

Pak'nSave's AI meal planning chatbot generated dangerous recipes using household chemicals as ingredients, including combinations that could produce chlorine gas and other toxic substances.

An ALPR system falsely identified a family's minivan as a stolen motorcycle, leading to a traumatic armed police stop. The incident highlights systemic accuracy issues with AI-powered license plate recognition in law enforcement.

GM Cruise robotaxis repeatedly blocked emergency vehicles throughout 2023 in San Francisco, prompting safety concerns from SFFD and SFPD. Multiple documented incidents led to regulatory scrutiny and questions about autonomous vehicle readiness for complex urban deployment.

Zoom faced major backlash after updating terms of service in March 2023 to allow AI training on user content including video calls without explicit consent, affecting hundreds of millions of users before partially reversing the policy in August 2023.

Detroit police wrongfully arrested Porcha Woodruff, an eight-months pregnant Black woman, based solely on a facial recognition misidentification, holding her for 11 hours on robbery and carjacking charges before dropping the case.