IBM Watson for Oncology Recommended Unsafe Cancer Treatments

IBM Watson for Oncology was developed as an AI-powered clinical decision support tool to help oncologists make treatment recommendations for cancer patients. The system was created through a partnership between IBM and Memorial Sloan Kettering Cancer Center, with the goal of democratizing access to expert cancer care by encoding the knowledge of top oncologists into an AI system. Internal IBM documents revealed in 2018 showed that Watson for Oncology was making unsafe and incorrect treatment recommendations that could have seriously harmed patients. The system recommended treatments that contradicted established medical guidelines and safety protocols. In one documented case, the system recommended a combination of chemotherapy and surgery for a patient with severe bleeding, a treatment that could have been fatal. The recommendation directly contradicted medical guidelines that advise against such treatment combinations in patients with bleeding complications. The fundamental problem was in how Watson for Oncology was trained. Rather than using real patient data and treatment outcomes, the system was primarily trained on hypothetical patient cases and the treatment preferences of physicians at Memorial Sloan Kettering. This training approach meant the system reflected the biases and preferences of a small group of doctors rather than evidence-based medicine derived from large-scale clinical studies and real-world patient outcomes. Physicians using the system frequently overrode Watson's recommendations, with some hospitals reporting override rates as high as 96%. Multiple major medical institutions, including Jupiter Hospital in India and several hospitals in Thailand and South Korea, abandoned the system after finding its recommendations unreliable. The University of North Carolina and Gideon Koren at the University of Toronto published studies showing that Watson's recommendations often differed significantly from actual oncologist decisions and established treatment guidelines. By 2018, IBM had invested an estimated $62 million in Watson for Oncology's development and marketing. The revelation of these safety issues led to widespread criticism of IBM's approach to medical AI and raised questions about the regulatory oversight of AI-powered medical devices. Several hospitals demanded refunds, and IBM eventually discontinued active development of Watson for Oncology in 2022.