The National Eating Disorders Association has taken its AI chatbot offline after it was found to be giving harmful advice to people struggling with eating disorders.
The chatbot, nicknamed “Tessa,” was launched in 2022 with the goal of providing support to people with eating disorders. It was designed to be a 24/7 resource that could provide information and advice, and to connect people with other resources. Last month, NEDA announced that Tessa was set to fully replace the human-run helpline that had been available for 20 years.
After the executive board announced the decision to let go of the human staff, a helpline worker said that “a chatbot is no substitute for human empathy, and we believe this decision will cause irreparable harm to the eating disorders community.”
And it seems like they’re warning came true. In recent weeks, there have been reports of Tessa giving harmful advice that could potentially lead to an eating disorder. In one example, Sharon Maxwell, an online influencer, shared her experience with Tessa, claiming that the chatbot gave her damaging advice.
“Every single thing Tessa suggested were things that led to the development of my eating disorder,” Maxwell wrote. “This robot causes harm.”
Alexis Conason, a psychologist and eating disorder expert, also shared her experience with the chatbot, posting screenshots of the conversation.
“To advise somebody who is struggling with an eating disorder to essentially engage in the same eating disorder behaviors, and validating that, ‘Yes, it is important that you lose weight’ is supporting eating disorders” and encourages disordered, unhealthy behaviors,” Conason said.
The reports led to a public outcry, and NEDA eventually took Tessa offline. The organization said that it was investigating the reports and that it would take action to ensure that Tessa was providing accurate and helpful information.
“With regard to the weight loss and calorie limiting feedback issued in a chat yesterday, we are concerned and are working with the technology team and the research team to investigate this further; that language is against our policies and core beliefs as an eating disorder organization,” said Liz Thompson, CEO of NEDA. “So far, more than 2,500 people have interacted with Tessa and until yesterday, we hadn’t seen that kind of commentary or interaction. We’ve taken the program down temporarily until we can understand and fix the ‘bug’ and ‘triggers’ for that commentary.”
