AI Chatbot GPT-3 Equals Undergraduates in SAT Problem Solving, Reveals UCLA Study

JJohn July 31, 2023 8:12 PM

A recent study reveals that artificial intelligence chatbot, GPT-3, performs on par with undergraduates in solving SAT problems. The study conducted at the University of California, Los Angeles (UCLA) examines the problem-solving abilities of GPT-3, throwing light on AI's progress in certain specific areas.

ChatGPT's undergraduate-level problem-solving

ChatGPT, built on the GPT-3 large language model, shows impressive problem-solving capabilities, matching or even surpassing an undergraduate student's level. A recent study has found the AI bot to perform about as well as a US college undergraduate in solving reasoning problems that appear on intelligence tests or exams such as the American college admission test, the SAT.

GPT-3 outperforms humans in problem-solving

Psychologists at the University of California, Los Angeles tested GPT-3’s ability to predict the next image in a complex array of shapes. The same problems were given to 40 UCLA undergraduates for comparison. The results showed that GPT-3 solved 80% of the problems correctly, which is well above the average score of just below 60% for the human participants.

AI surpasses human average on SAT analogies

Further, the researchers challenged the AI to solve some SAT 'analogy' questions – selecting pairs of words that are linked in some way – which they believe had not been published on the internet, ensuring the model's unfamiliarity with the questions. When the AI's performance was compared with college applicants' SAT scores, it was found to outperform the average score for humans.

However, the AI did not perform as well in all areas. When the researchers asked GPT-3 and the student volunteers to match a passage of prose with a different short story that conveyed the same meaning, the AI did not perform as well as the students. But GPT-4, the improved successor to GPT-3, showed an enhanced performance, doing better than its predecessor.

ChatGPT's struggles and strides

Despite its commendable performance, the AI model driving ChatGPT is not at the standard of artificial general intelligence or human-level intelligence. The model struggles with tasks involving social interactions, mathematical reasoning, and understanding physical space. However, it marks a significant advancement in specific areas of AI progress.

The UCLA researchers pointed out that without access to the ins and outs of GPT-3, developed by San Francisco-based company OpenAI, it is challenging to ascertain how the model's reasoning abilities function. They are unsure if it thinks like a human or if it's showing a new form of intelligence. This poses an interesting question about the true nature of artificial intelligence and its future implications.

More articles

Also read

Here are some interesting articles on other sites from our network.