AI Incident Database

LSEG's World-Check screening database used AI algorithms that falsely flagged hundreds of innocent people as terrorists or criminals, causing them to be denied banking services and face reputational harm.

AI fraud detection systems at Chase, Wells Fargo, and Bank of America systematically froze legitimate customer accounts, disproportionately affecting minorities and gig workers, leading to class action lawsuits and FTC investigation.

AI translation tools used by U.S. border control agents produced incorrect translations of Arabic and other languages, leading to wrongful detentions of travelers based on false criminal associations created by translation errors.

Multiple AI photo editing tools were discovered removing or altering people of color from images when users requested photo enhancement, revealing systematic bias in training data and beauty standard algorithms.

An ALPR system falsely identified a family's minivan as a stolen motorcycle, leading to a traumatic armed police stop. The incident highlights systemic accuracy issues with AI-powered license plate recognition in law enforcement.

Detroit police wrongfully arrested Porcha Woodruff, an eight-months pregnant Black woman, based solely on a facial recognition misidentification, holding her for 11 hours on robbery and carjacking charges before dropping the case.

AI writing detection tools at UC Davis and other universities systematically flagged writing by non-native English speakers as AI-generated, leading to false academic dishonesty accusations and highlighting significant bias in content detection technology.

Turnitin's AI detection tool falsely flagged human-written essays as AI-generated, disproportionately affecting non-native English speakers and leading to wrongful academic discipline and multiple lawsuits against universities.

NYC's Local Law 144 mandating bias audits of automated hiring tools revealed significant demographic disparities in AI screening systems. Multiple companies' tools showed substantially different selection rates across protected groups, highlighting systemic bias in employment algorithms.

A 2023 NIJ study found that AI risk assessment tools used in 40+ states systematically assigned higher risk scores to Black defendants. These tools influence parole, sentencing, and pre-trial detention decisions affecting hundreds of thousands annually.

iTutorGroup's AI hiring tool automatically rejected older applicants, leading to a $365,000 EEOC settlement in 2023 for age and gender discrimination.

Multiple studies revealed AI resume screening tools systematically discriminated against disabled job applicants by penalizing employment gaps and accommodation keywords, prompting EEOC guidance on AI hiring discrimination.

iTutorGroup's AI hiring software systematically rejected female applicants over 55 and male applicants over 60, resulting in the first major EEOC settlement for AI-driven employment discrimination at $365,000.

AI writing detection tools like Turnitin and GPTZero produced widespread false positives in 2023, incorrectly flagging human-written student work as AI-generated. Non-native English speakers were disproportionately affected, leading to unfair disciplinary actions.

Paradox's AI recruiting chatbot Olivia was accused of age discrimination through interface design and language patterns that systematically disadvantaged older job applicants who were less familiar with digital communication.

AI surveillance towers deployed by CBP along the US-Mexico border generated false alerts flagging legitimate asylum seekers as threats. The systems demonstrated algorithmic bias, leading to unnecessary detentions and civil rights concerns.

Paradox's AI recruiting chatbot Olivia was reported to provide less responsive service to candidates with names suggesting certain ethnic backgrounds, raising discrimination concerns in automated hiring processes.

AI recruitment systems systematically penalized job candidates with employment gaps from medical leave, military service, and caregiving responsibilities, leading to discriminatory hiring practices affecting thousands of applicants.

Macy's facial recognition system falsely identified a Black customer as a shoplifter, leading to wrongful detention and public humiliation in Houston, highlighting ongoing issues with AI bias in retail security.

A 2023 class action lawsuit alleged that Workday's AI-powered hiring screening tools systematically discriminated against older workers and disabled applicants, marking a significant case targeting the HR technology vendor rather than just employers.