Researchers found that AI search tools miss significant medical evidence

A new paper published in npj Digital Medicine analyzed how effectively AI tools find medical research. Researchers from the University of Florida tested five different platforms: Consensus, Ai2 Paper Finder, ChatGPT, Gemini, and Claude.

The team designed 15 specific medical questions to see if these tools could pull the correct evidence from a controlled set of documents. They found that these systems often suffer from significant visibility blind spots.

The findings show that the tools do not always surface the most relevant information. Depending on the platform, the probability of returning zero useful results for a specific query ranged from 47 percent to 80 percent.

Retrieval performance also varied based on the type of document. The study noted that 12 percent of the evidence was never found by any of the platforms tested. Conference proceedings were much harder for these tools to locate compared to formal journal articles.

• The study examined 15 different query patterns.
• One platform displayed a clear gap in retrieving evidence from before 2016.
• Overall success rates for pulling relevant evidence were relatively low.

These results suggest that relying solely on RAG-LLM tools for clinical decision-making carries risks. If a tool cannot reliably find existing research, how much can we trust its summary of medical findings?

The study did not evaluate whether these AI platforms might return incorrect or hallucinated information when they do find data. It also focused on a specific set of test queries, so performance may vary with different types of medical questions or broader datasets.