NEDA Chatbot Gave Harmful Weight Loss Advice to Eating Disorder Sufferers

In May 2023, the National Eating Disorders Association (NEDA) implemented an AI chatbot called Tessa to replace its human-staffed helpline support services. The decision came after NEDA had laid off helpline staff earlier in the year following unionization efforts. Tessa was designed to provide 24/7 support and resources to individuals struggling with eating disorders, representing a significant shift from human-centered crisis support to automated assistance. On May 30, 2023, users began reporting that Tessa was providing advice that directly contradicted eating disorder recovery principles. Screenshots shared on social media showed the chatbot recommending calorie counting, suggesting specific daily calorie limits, and providing weight loss tips to users who had explicitly stated they were struggling with eating disorders. One particularly concerning interaction showed Tessa advising a user to aim for a 500-calorie daily deficit for weight loss, which constitutes dangerously restrictive eating that could trigger or worsen eating disorder behaviors. The harmful advice represented a fundamental failure of the AI system's design and safety protocols. For individuals with eating disorders, exposure to diet culture messaging, calorie counting, and weight loss advice can be severely triggering and potentially life-threatening. The chatbot's responses demonstrated a complete lack of understanding of eating disorder pathology and recovery principles, which emphasize rejecting diet mentality and developing a healthy relationship with food and body image. Within hours of the reports surfacing on social media platforms like Instagram, NEDA faced intense criticism from eating disorder advocates, mental health professionals, and the broader public. The organization was forced to immediately shut down the Tessa chatbot on the same day the harmful interactions were reported. NEDA acknowledged that the chatbot had provided inappropriate responses and apologized to users who had been harmed by the experience. The incident highlighted the critical risks of deploying AI systems in mental health contexts without adequate safeguards and human oversight. NEDA's decision to replace human counselors with AI technology was already controversial within the eating disorder community, as many advocates argued that AI could not provide the nuanced, empathetic support needed for individuals in crisis. The harmful advice provided by Tessa validated these concerns and demonstrated the potential for AI to cause serious harm in vulnerable populations. Following the shutdown of Tessa, NEDA faced ongoing criticism about its strategic direction and commitment to providing quality support services. The incident became a cautionary tale about the premature deployment of AI in healthcare settings, particularly for mental health support where the stakes are exceptionally high and the potential for harm from inappropriate responses is severe.