In a study of 2,000 chest X-rays, radiologists proved more accurate than AI in detecting common lung diseases. AI achieved sensitivity rates similar to radiologists but produced more false positives, especially in complex cases. For pneumothorax, AI’s positive predictive values were lower. Researchers stressed the importance of AI complementing radiologists rather than replacing them, emphasizing the need for AI to synthesize clinical history. This study underscores that radiologists generally outperform AI in diverse clinical scenarios, where patients often have multiple conditions. AI can serve as a valuable second opinion for radiologists.
A recent study involving over 2,000 chest X-rays has revealed that radiologists surpass AI in accurately identifying three common lung diseases. This research was published in the Radiology journal, affiliated with the Radiological Society of North America (RSNA).
Lead researcher Dr. Louis L. Plesner, a resident radiologist and Ph.D. fellow at the Department of Radiology at Herlev and Gentofte Hospital in Copenhagen, Denmark, emphasized the necessity of significant training and experience for correct interpretation of chest X-rays. While AI tools, approved by the FDA and available for commercial use, aim to assist radiologists, Dr. Plesner emphasized that their clinical application for radiological diagnoses is still in its early stages.
Dr. Plesner also pointed out the importance of further testing AI tools in real clinical scenarios, stating that although AI can assist radiologists in interpreting chest X-rays, their real-world diagnostic accuracy remains uncertain.
In this study, Dr. Plesner and his team compared the performance of four commercially available AI tools with 72 radiologists in assessing 2,040 adult chest X-rays taken over two years at four Danish hospitals in 2020. The patients had a median age of 72 years, and 32.8% of the X-rays showed at least one target finding.
The chest X-rays were evaluated for three common findings: airspace disease (such as pneumonia or lung edema), pneumothorax (collapsed lung), and pleural effusion (fluid buildup around the lungs).
AI tools achieved sensitivity rates ranging from 72% to 91% for airspace disease, 63% to 90% for pneumothorax, and 62% to 95% for pleural effusion. While the AI tools demonstrated moderate to high sensitivity comparable to radiologists in detecting these conditions, they generated more false-positive results than the radiologists, especially when multiple findings were present or for smaller targets.
For pneumothorax, the positive predictive values of the AI systems ranged from 56% to 86%, in contrast to 96% for the radiologists.
Dr. Plesner highlighted that AI performed the least effectively in identifying airspace disease, with positive predictive values ranging from 40% to 50%. He emphasized the importance of not solely relying on AI systems, especially when dealing with challenging patient samples, as they tended to predict the presence of airspace disease when it was not present, about five to six out of 10 times.
In the context of radiology, the goal is to strike a balance between detecting and excluding diseases, avoiding both missed diagnoses and overdiagnosis. AI systems, according to Dr. Plesner, excel at finding diseases but fall short of radiologists in identifying the absence of disease, especially in complex chest X-rays. An excess of false-positive diagnoses could lead to unnecessary imaging, radiation exposure, and increased healthcare costs.
Dr. Plesner noted that most studies have focused on AI’s ability to identify the presence or absence of a single disease, which is less challenging than the real-world scenarios where patients often have multiple conditions. He emphasized the need for AI systems that can synthesize clinical history and previous imaging studies, a capability that does not currently exist in AI systems.
Overall, Dr. Plesner emphasized that radiologists generally outperform AI in real-world scenarios with diverse patient cases. While AI can effectively identify normal chest X-rays, it should not be solely relied upon for making diagnoses. Instead, these AI tools can enhance radiologists’ confidence by providing a second opinion on chest X-ray interpretations.
