Amazon Rekognition Facial Recognition System Exhibited Racial Bias in Congressional Test

In July 2018, the American Civil Liberties Union (ACLU) conducted a test of Amazon's Rekognition facial recognition system that revealed significant racial bias in the technology. The ACLU compared official photos of all 535 members of Congress against a database of 25,000 publicly available arrest photos using Amazon's default confidence threshold of 80%. The test resulted in 28 false matches, incorrectly identifying sitting members of Congress as individuals who had been arrested. The results demonstrated a clear pattern of racial bias: while people of color make up only 20% of Congress, they represented 39% of the false matches. This disproportionate impact was particularly concerning given that Amazon was actively marketing Rekognition to law enforcement agencies across the United States. The false identifications included prominent members of Congress from diverse backgrounds, highlighting how algorithmic bias could lead to wrongful arrests or harassment of innocent individuals. Amazon responded by disputing the ACLU's methodology, arguing that the 80% confidence threshold used was too low for law enforcement applications and that their guidelines recommended a 95% threshold for such sensitive use cases. However, the company's own documentation and marketing materials had not clearly emphasized these limitations, and many law enforcement agencies were already using the system with default settings. The incident revealed a disconnect between Amazon's internal technical knowledge and how the system was being marketed and deployed. The revelation sparked widespread criticism from civil rights groups, researchers, and policymakers. MIT researcher Joy Buolamwini had previously documented similar bias patterns in facial recognition systems from multiple vendors, showing that these technologies consistently performed worse on women and people with darker skin tones. The Congressional test provided a high-profile demonstration of how these technical limitations translated into real-world discrimination risks. The incident contributed to growing scrutiny of facial recognition technology and ultimately led to policy changes. Several major cities banned law enforcement use of facial recognition, and Amazon eventually implemented a moratorium on police use of Rekognition in 2020. The company has since required higher confidence thresholds and implemented additional bias testing, though critics argue these measures are insufficient to address fundamental algorithmic fairness issues.