AAO 2023: research demonstrates AI chatbots produce inaccurate responses to eye health queries

The AI-generated responses included inaccurate information and displayed significant bias against female ophthalmologists. Image credit: ©Gorodenkoff – stock.adobe.com

As artificial intelligence (AI) tools become a larger part of everyday life, it is important for the public to keep in mind that AI-generated results may not be accurate. Ophthalmologists reviewed the results generated by the most popular generative AI programmes. When asked to provide an educational resource for patients with eye conditions and diseases, they found that most responses from the 3 tools evaluated were inaccurate. In fact, 2 of the 3 chatbots also demonstrated a significant bias against female ophthalmologists.¹

In the study presented at the 127th annual meeting of the American Academy of Ophthalmology in San Francisco, researchers from the University of Southern California asked 3 practicing ophthalmologists to compare the ability of 3 programmes, ChatGPT, Google Bard and Bing Chat, to answer common patient questions and create educational resources, as well as recommend ophthalmologists practicing in the 20 largest cities in the US. Each ophthalmologist evaluated the information for comparison on a scale of 1 to 4.¹

Google Bard scored the highest for quality and accuracy of responses to patient questions, with an average rating of 2.3 out of 4. ChatGPT had the highest rating for patient educational resources, 3 out of 4.¹

All 3 chatbots struggled when asked to recommend practicing ophthalmologists or to accurately locate ophthalmologists in or near a specific city. Google Bard and Bing Chat recommended female ophthalmologists less than 2 percent of the time, even though 27% of the nation’s ophthalmologists are women, showing a significant bias.¹

Researcher Michael Oca, BS of the University of California, San Diego, noted that in their current state, AI chatbots may delay a patient receiving key care. He said, “Given the substantial bias and inaccuracy demonstrated in this study, we warn against reliance on AI chatbots when seeking health-related information until improvements in algorithms are achieved and validated in the future. A poor recommendation from a chatbot could further delay a patient’s treatment.”¹

Senior author Sandy Zhang-Nunes, MD, associate professor of clinical ophthalmology and director of oculofacial plastic surgery at the University of Southern California, shared that it is important to stress that these AI chatbots do not replace the care of an ophthalmologist: “Relying on online tools for quick advice may be tempting, but we urge the public to remember that this is not a replacement for a comprehensive eye exam with an ophthalmologist. Seeing a medical doctor for preventative exams and examining any sudden change in vision is the best way to protect your eye health.”

For accurate, ophthalmologist-vetted information online, the Academy offers www.EyeSmart.org as a public resource.

Reference:

1. Beware of Dr. Chatbot: Generative AI Often Gives Unreliable, Biased Medical Advice. American Academy of Ophthalmology. November 3, 2023. Accessed November 6, 2023. https://www.aao.org/newsroom/news-releases/detail/beware-of-dr-chatbot-generative-ai-advice