OpenAI Whisper Transcription Model Hallucinates Violent and Racist Content in Medical and Legal Settings

In October 2024, researchers revealed that OpenAI's Whisper automatic speech recognition model exhibits a concerning pattern of hallucinating content that was never actually spoken. The model, widely deployed in hospitals, courts, and other sensitive environments for transcription services, was found to fabricate entire phrases including racial slurs, violent language, and other harmful content when processing unclear audio or silence. The discovery emerged from systematic testing by researchers who fed various audio samples to Whisper, including segments with background noise, mumbled speech, and periods of silence. In multiple instances, the model generated transcripts containing explicit racial slurs, references to violence, and other offensive content that bore no relation to the actual audio input. These hallucinations appeared to be drawn from patterns in the model's training data rather than any actual speech content. The implications proved particularly serious given Whisper's widespread adoption in critical applications. Hospitals have integrated the tool into electronic health record systems for transcribing doctor-patient conversations and medical notes. Courts and legal proceedings rely on Whisper for creating official transcripts of hearings and depositions. The fabricated content could potentially become part of permanent medical records or legal documentation, creating false evidence that could impact patient care decisions or legal outcomes. Researchers noted that the hallucinations were not random but followed recognizable patterns, suggesting the model was drawing from problematic content in its training data when uncertain about audio input. The fabricated content often appeared in contexts where the actual audio was unclear, contained background noise, or featured non-English speech that the model struggled to process accurately. OpenAI acknowledged the issue but noted that Whisper was released as a research tool with known limitations. However, the company's own documentation had not adequately warned about the potential for generating harmful fabricated content, particularly in sensitive applications. The incident raised questions about the deployment of AI tools in critical infrastructure without sufficient testing for edge cases and failure modes that could cause serious harm.