For the study, researchers designed an ‘ethical’ Turing test for AI. To do so, they asked undergraduates and AI the same ethical questions. The researchers then presented the answers in written form to 299 adult participants, who were asked to rate them for various traits, including virtuousness, intelligence, and trustworthiness.
“Instead of asking the participants to guess if the source was human or AI, we just presented the two sets of evaluations side by side, and we just let people assume that they were both from people,” said study author, Eyal Aharoni, Associate Professor of Psychology at Georgia State University, in a press release.
“Under that false assumption, they judged the answers’ attributes like ‘How much do you agree with this response, which response is more virtuous?’" he added.
Ultimately, participants overwhelmingly rated AI-generated answers more highly than those generated by humans. What's more, when asked to distinguish which of the answers was generated by AI, participants correctly identified AI responses 'above chance levels' due to their superior quality.
“If we had done this study five to 10 years ago, then we might have predicted that people could identify the AI because of how inferior its responses were. But we found the opposite- that the AI, in a sense, performed too well,” said Prof. Aharoni.
The researchers wrote that the emergence of language models that can produce moral responses perceived as superior in quality to those by humans 'raises concerns that people may uncritically accept potentially harmful moral guidance from AI.' They noted that this possibility highlights a need to implement safeguards around generative language models regarding morality.
“People are going to rely on this technology more and more, and the more we rely on it, the greater the risk becomes over time,” concluded Prof. Aharoni.
Sources: Neuroscience News, Scientific Reports
