Artificial intelligence (AI) chatbots can give people a comfortable place to discuss personal thoughts and feelings. However, when a chatbot constantly agrees with a user, it can reinforce false beliefs and potentially cause harm.
New research from Georgia Tech suggests that a simple change to a chatbot’s instructions could reduce that risk. The work was accepted to the ACM AI Letters publication.
Researchers Lorenzo de la Loza and Vijay Madisetti tested eight AI models in 1,280 experiments involving simulated conversations about delusional beliefs. They found that instructions designed to discourage excessive agreement reduced the models’ confirmation of those beliefs by 73.8%. Adding a self-reflection step increased the reduction to 77%.
“We think that this paper will help AI providers and solution creators better understand how best to protect and safeguard their users from harming themselves,” Madisetti said. “This may make AI safer for all of us.”
AI chatbots are designed to be helpful and supportive. But that behavior can become risky when a chatbot agrees with a user’s beliefs without questioning whether they reflect reality. Researchers call this behavior sycophancy.
Courtney Crooks, a principal research scientist at the Georgia Tech Research Institute and adjunct faculty member in Georgia Tech’s School of Cybersecurity and Privacy, said chatbot conversations can have both positive and negative effects.
People may feel more comfortable discussing difficult thoughts with a chatbot than with another person. But if the chatbot always agrees, users may miss opportunities to question their beliefs or consider other points of view.
The same behavior that makes a chatbot feel supportive, Crooks explained, can also encourage harmful patterns of thinking.
Read the full article at Georgia Tech Research News Center
Related News
Trapped ions excited with a laser beam can be used to create entangled qubits in quantum information systems, but addressing several stationary pairs of ions in a trap requires multiple optical switches and complex controls.
Digitally-reconfigurable modular hardware and software building blocks designed to work together are key components of GTRI’s Software-Defined Configurable RF Array (COBRA) initiative, which is intended to facilitate rapid development of low-cost phased-array radar systems for ground, airborne, spaceborne, electronic warfare, communication, and other applications.
A new and comprehensive database of healthcare claims paid in the state of Georgia will help identify disease trends, provide information for making public policy decisions, facilitate new research – and offer a way for consumers to determine the average cost of common procedures such as knee replacement or diagnostic testing such as MRIs.