ChatGPT Surpasses WebMD in Identifying Eye-related Complaints

NNicholas September 16, 2023 2:51 PM

Researchers from Emory University School of Medicine have found that the AI chatbot, ChatGPT, outperforms popular health website WebMD at diagnosing eye-related complaints. Despite initial skepticism and known issues with the AI model, ChatGPT demonstrated superior accuracy and made no 'grossly inaccurate' statements during the study, prompting discussions on its wider integration into healthcare systems.

ChatGPT's accuracy in diagnosing eye complaints

As AI enters various sectors of society, the healthcare industry also looks to harness its potential. Researchers at Emory University School of Medicine utilized the AI chatbot, ChatGPT to diagnose eye-related complaints. Their findings, published on medRxiv, suggest that ChatGPT performed admirably, outdoing the symptom checker on the popular health website, WebMD. Despite ChatGPT's known 'hallucination' problem, where it makes outright false statements, the study reported that it made zero 'grossly inaccurate' statements when presented with a standard set of eye complaints.

The unexpected proficiency of ChatGPT, which debuted in November 2022, took the researchers by surprise. Its superior performance over a simple Google search emphasizes the potential of AI in healthcare. However, this also puts forward a challenge for the healthcare industry. Although the accuracy of chatbot-delivered medical information is an improvement over 'Dr. Google', questions arise on how to integrate this new technology into healthcare systems with the same safeguards typically applied to the introduction of new drugs or medical devices.

The unique capabilities of generative AI, such as its smooth syntax, authoritative tone, and versatility, have garnered immense attention from all sectors of society. In the healthcare sector, companies are tirelessly working to implement generative AI in various areas, including radiology and medical records. However, the integration of consumer chatbots into healthcare still demands a cautious approach, despite the technology's wide availability and superiority over many alternatives.

Calls for regulatory oversight for AI-based medical tools

Despite the evident benefits, many medical professionals express concern over the hasty integration of AI-based medical tools. They argue that such tools should go through an approval process akin to the FDA's regime for drugs. They highlight issues related to privacy, safety, bias, liability, transparency, and the current absence of regulatory oversight. In the absence of a concrete framework, it remains unclear how such regulations might apply to general-purpose AIs like ChatGPT.

Public uptake and potential pitfalls of AI tools

The fact that AI tools like ChatGPT are already in use by a large number of people underlines their potential and raises urgent questions about future regulation. There's a growing recognition that these tools offer new capabilities to both physicians and patients. That said, even researchers who have demonstrated ChatGPT's relative reliability advise against patients putting their full trust in the current state of AI, citing potential pitfalls such as the exploitation of data and the possible commercialization of personal information.

More articles

Also read

Here are some interesting articles on other sites from our network.