Allegheny County Child Welfare AI Tool Showed Racial Bias in Family Risk Assessments

In 2016, Allegheny County, Pennsylvania implemented the Allegheny Family Screening Tool (AFST), an algorithmic risk assessment system designed to help child welfare workers prioritize which referrals to investigate. The system analyzed over 100 variables from county databases, including child welfare history, criminal records, jail bookings, mental health services, and public benefit usage, to generate risk scores for families reported to child protective services. A comprehensive investigation by the Associated Press published in April 2022 revealed significant racial disparities in the tool's operation. The analysis found that Black families were flagged at disproportionately high rates compared to white families with similar circumstances. The investigation examined data from 2016 to 2022, covering approximately 50,000 referrals processed through the system. Families receiving public benefits like food stamps and Medicaid were also more likely to receive high-risk scores, creating a proxy for discrimination based on socioeconomic status. The algorithmic bias stemmed from the system's reliance on historical child welfare data that reflected decades of systemic discrimination in family services. Additionally, the inclusion of variables like public benefit usage, prior involvement with county services, and neighborhood-level factors created pathways for racial and economic bias to influence risk assessments. Research by academic partners found that these proxy variables effectively encoded racial disparities into the algorithm's decision-making process. The county defended the tool, arguing that it helped workers manage heavy caseloads and that all referrals received human review. Officials emphasized that AFST was a screening tool, not a decision-making system, and that caseworkers retained ultimate authority over investigation decisions. However, critics noted that busy social workers likely relied heavily on the algorithmic recommendations, making the bias practically consequential for families' lives. The incident highlighted broader challenges with predictive risk assessment tools in child welfare, where algorithmic efficiency can amplify historical injustices. Following the AP investigation, advocacy groups called for greater transparency and accountability in algorithmic decision-making systems used by government agencies. The case became a prominent example of how well-intentioned AI systems can perpetuate and systematize existing biases when deployed without adequate safeguards.