Lunit recently announced the results of its study demonstrating the effectiveness of artificial intelligence (AI) solutions in cancer diagnostics.
The study, published in the journal Radiology, explored the impact of medical AI solutions’ accuracy on radiologists’ diagnostic determination. It used Lunit’s Insight CXR AI solution for chest X-ray analysis and showed that AI could improve radiologists’ performance.
There were 30 doctors involved in the study, including 20 board-certified radiologists with five to 18 years of expertise and 10 radiology residents with two to three years of training. The study assessed a total of 120 chest radiographs were assessed, with 60 obtained from patients with lung cancer and the remaining 60 showing no abnormalities. Seoul National University Hospital conducted the study from December 2015 to February 2021.
During the first session, 30 readers were divided into two groups and analyzed 120 chest X-rays each without the assistance of AI. In the subsequency session, each group reinterpreted the images with the aid of a high-accuracy or low-accuracy AI model.
More about the study results
The study’s high-accuracy AI model utilized Lunit’s Insight CXR, while the low-accuracy model was trained using only 10% of the data available to Lunit’s Insight CXR. The AUROC (area under the receiver operating characteristic curve), a commonly used metric for diagnostic accuracy, of Lunit Insight CXR was 0.88, while the low-accuracy AI model only reached 0.77.
According to the study, Lunit’s higher-accuracy AI model significantly improved radiologists’ performance. The company said the AUROC was “remarkably advanced” from 0.77 to 0.82 when assisted by the high-accuracy AI model.
Radiologists in the other group did not experience any performance improvement when utilizing the low-accuracy AI model, as AUROC remained at 0.75. The group using the high-accuracy AI model was more susceptible to AI suggestions. The radiologists accepted 67% of AI recommendations that contradicted the initial reading results, compared to 59% acceptance of the group that utilized the low-accuracy AI model.
The study findings highlighted that factors such as radiologists’ individual expertise, experience with AI or attitudes toward AI had negligible impact on their reading performance in the second session. The accuracy of the AI model and the radiologists’ initial diagnostic accuracy emerged as the primary determinant shaping the final diagnostic determination.
“The study backs that irrespective of radiologists’ individual characteristics, the utilization of high-performance AI significantly enhances diagnostic accuracy and fosters a greater acceptance of AI within medical practices,” CEO Brandon Suh said in a news release. “At Lunit, we are committed to developing AI-powered solutions that not only improve patient outcomes but also augment the expertise of healthcare professionals. This publication is a testament to our dedication to advancing the field of cancer diagnostics through cutting-edge technology.”