Deep Learning-Based ASPECTS Algorithm Enhances Reader Performance and Reduces Interpretation Time

BACKGROUND AND PURPOSE: ASPECTS is a long-standing and well documented selection criteria for acute ischemic stroke treatment, however, the interpretation of ASPECTS is a challenging and time-consuming task for physicians with significant interobserver variabilities. We conducted a multi-reader, multi-case study in which readers assessed ASPECTS without and with the support of a deep learning (DL)-based algorithm in order to analyze the impact of the software on clinicians' performance and interpretation time.

MATERIALS AND METHODS: A total of 200 NCCT scans from 5 clinical sites (27 scanner models, 4 different vendors) were retrospectively collected. Reference standard was established through the consensus of three expert neuroradiologists who had access to baseline CTA and CTP data. Subsequently, eight additional clinicians (four typical ASPECTS reader and four senior neuroradiologists) analyzed the NCCT scans without and with the assistance of CINA-ASPECTS (Avicenna.AI, La Ciotat, France), a DLbased FDA-cleared and CE-marked algorithm designed to automatically compute ASPECTS. Differences were evaluated in both performance and interpretation time between the assisted and unassisted assessments.

RESULTS: With software aid, readers demonstrated increased region-based accuracy from 72.4% to 76.5% (p<0.05), and increased ROC AUC from 0.749 to 0.788 (p<0.05). Notably, all readers exhibited an improved ROC AUC when utilizing the software. Moreover, use of the algorithm improved the score-based inter-observer reliability and correlation coefficient of ASPECTS evaluation by 0.222 and 0.087 (p<0.0001), respectively. Additionally, the readers’ mean time spent analyzing a case was significantly reduced by 6% (p<0.05) when aided by the algorithm.

CONCLUSIONS: With the assistance of the algorithm, readers' analyses were not only more accurate but also faster. Additionally, the overall ASPECTS evaluation exhibited greater consistency, less variabilities and higher precision compared to the reference standard. This novel tool has the potential to enhance patient selection for appropriate treatment by enabling physicians to deliver accurate and timely diagnosis of acute ischemic stroke.

ABBREVIATIONS: ASPECTS = Alberta Stroke Program Early Computed Tomography Score; DL = Deep Learning; EIC = Early Ischemic Changes; ICC = Intraclass Correlation Coefficient; IS = Ischemic Stroke; ROC AUC = Receiver Operating Characteristics Area Under the Curve.