AI Incident Database

Stanford research found GitHub Copilot generates more security vulnerabilities than human-written code while causing developers to over-trust AI suggestions, raising concerns about enterprise software security.

X's Grok AI chatbot generated false election information in 2024, including incorrect voting details and fabricated results, prompting complaints from state officials and highlighting risks of training AI on unvetted social media content.

Meta's AI assistant on Facebook and Instagram fabricated having children in users' local schools, alarming parents and highlighting AI hallucination risks in social media contexts.

Meta's AI Studio allowed users to create Instagram chatbots impersonating real people without consent. The platform launched without adequate safeguards against identity fraud, affecting hundreds of individuals before Meta implemented restrictions.

xAI's Grok chatbot generated false election information in 2024, including wrong voting dates and fabricated candidate statements, raising concerns about AI misinformation during critical democratic processes.

xAI's Grok chatbot generated false election information including incorrect ballot deadlines and voting procedures, prompting intervention from election officials and highlighting risks of AI misinformation during critical democratic processes.

Perplexity AI faced accusations from Forbes and other publishers for scraping protected content, bypassing robots.txt restrictions, and generating fabricated source citations while providing inadequate attribution to original creators.

The RIAA and major record labels sued AI music companies Suno and Udio in 2024, alleging their generative models were trained on copyrighted music without permission and can reproduce existing songs.

In 2024, scammers used AI voice cloning technology to impersonate IRS agents, Medicare representatives, and law enforcement officials in sophisticated phone fraud campaigns. The FTC reported significant increases in government impersonation scams, with estimated losses in the tens of millions affecting thousands of victims nationwide.

McDonald's discontinued its IBM-developed AI drive-through ordering system after viral incidents showed it repeatedly misunderstanding orders and adding hundreds of dollars of unwanted items.

McDonald's ended its AI drive-through partnership with IBM in June 2024 after viral videos showed the system making absurd order errors, including adding hundreds of unwanted items to customer orders.

Microsoft's AI Recall feature stored unencrypted screenshots of all user activity including passwords and sensitive data, forcing the company to delay launch after major security backlash.

Microsoft's Recall feature for Copilot+ PCs stored unencrypted screenshots of user activity in accessible databases, creating massive privacy risks. Security researchers' findings led to public backlash and Microsoft delaying the feature.

NewsGuard identified over 1,000 AI-generated fake local news websites producing fabricated articles for political propaganda and ad fraud, undermining trust in legitimate journalism and democratic discourse.

NYC's AI-powered MyCity chatbot gave illegal advice to small businesses, including telling landlords they could discriminate based on income source and advising minimum wage violations.

Google's AI Overviews feature generated dangerous health advice including eating rocks and using glue on pizza, sourcing information from satirical Reddit posts without quality filtering.

Google's AI Overviews feature provided dangerous advice including eating rocks for minerals and using glue on pizza, sourced from satirical content. Google quickly scaled back the feature after widespread criticism.

Meta's AI detection system incorrectly labeled authentic photographs from the Gaza conflict as AI-generated in May 2024, undermining photojournalists' credibility and raising concerns about automated content moderation during sensitive news events.

Scarlett Johansson accused OpenAI of copying her voice for GPT-4o's 'Sky' assistant without permission, leading to legal threats and OpenAI pausing the voice feature.

OpenAI dissolved its Superalignment safety team in May 2024 after key safety leaders Jan Leike and Ilya Sutskever resigned, citing concerns that safety had taken a back seat to product development.