Researchers at the University of California-Los Angeles (UCLA) have shown in a recent study that ChatGPT, the artificial intelligence platform that took the world by storm earlier this year, that ChatGPT can perform nearly as well as college students on certain types of tests, including those found on standardized assessments like the SAT. The team’s research is published in Nature Human Behavior.
Specifically, researchers looked to compare how humans and ChatGPT solve problems. In many cases, people are capable of solving problems without any special type of training or learning, often by using prior experiences with solving problems to inform future problem solving. In many cases, this type of problem solving is something seemed uniquely human.
To compare human and ChatGPT capabilities, researchers asked ChatGPT and about 40 students from UCLA to complete a test called Raven’s Progressive Matrices. In these tests, a person is asked to make sequential predications, usually to predict the next image in a sequence of images. Researchers provided ChatGPT with text versions of the images so that it could “see” them.
Findings suggested that ChatGPT could perform about as well as a human could. ChatGPT was able to accurately predict the next image in a sequence about 80% of the time compared to about 60% among UCLA students. However, the 80% accuracy of ChatGPT was not beyond the upper limits of human capabilities.
They also asked ChatGPT to answer analogy questions, putting two similar words together. It was still able to outperform UCLA students. And when asked to read a passage of text and outline another text that conveyed the same meaning, ChatGPT3 was less successful, but the more recent version, ChatGPT4, was more successful.
One area where ChatGPT faltered, however, were tasks that required some sense of physical space. For example, when asked to describe how it would move an object to a different location using tools like a box, for example, ChatGPT proposed strange solutions that didn’t make sense.
What remains unclear, however, is if artificial intelligences like ChatGPT are employing a new type of analytical intelligence or simply imitating human reasoning using the vast data set it learns from. Without access to ChatGPTs inner workings, however, researchers may not be able to adequately answer this question.
Sources: Science Daily; Nature Human Behavior