Chatbot Tess of the trust service led to the development of a mental disorder in a patient
The company replaced all service employees with a machine, but the machine began to give out bad advice.
National Association for Eating Disorders (NEDA) blocked her chatbot Tessa for giving people bad advice.
Obesity activist and weight loss consultant Sharon Maxwell said that Tessa’s advice for a safe recovery from an eating disorder (EDD) is in direct conflict with medical advice.
A bot from an American non-profit organization recommended that Maxwell count calories, weigh herself weekly, and even suggested where to buy a caliper to measure body fat. By data NEDA, safe recovery is a multi-step process that includes many aspects of treatment and various therapies, not just calorie counting.
“Everything Tessa suggested led me to develop an eating disorder. This robot is causing harm.” stated Maxwell.
Earlier, a representative of the organization reported that Tessa had been tested on 700 volunteers since November 2021 and more than half of them gave the chatbot a 100% usefulness rating. However, something went wrong.
NEDA confirmed that disabled Tessa and investigated the chatbot’s advice. The organization said the current version of the Tessa chatbot, which runs the Body Positive program, may have provided information that was harmful and unrelated to the program.
It is worth noting that the Tessa robot appeared due to the fact that NEDA decided to replace employees of the psychological hotline with a chatbot . This happened exactly 4 days after some of the employees united in a trade union to demand better working conditions.
The NEDA Hotline has been in existence for 20 years and has provided support to hundreds of thousands of people via phone, chat and text messages. NEDA employees said they felt overwhelmed due to a lack of operators. The entire line was kept literally by six employees who received calls and messages from all American states. When employees asked the NEDA administration to recruit new people as well as improve their own working conditions, their demands were ignored. That is why they decided to organize a trade union.
Source link
www.securitylab.ru