Abstract
BACKGROUND AND PURPOSE: Time-resolved MRA enables collateral evaluation in acute ischemic stroke with large-vessel occlusion; however, a low SNR and spatial resolution impede the diagnosis of vascular occlusion. We developed a CycleGAN-based deep learning model to generate high-resolution synthetic TOF-MRA images using time-resolved MRA and evaluated its image quality and clinical efficacy.
MATERIALS AND METHODS: This retrospective, single-center study included 397 patients who underwent both TOF- and time-resolved MRA between April 2021 and January 2022. Patients were divided into 2 groups for model development and image-quality validation. Image quality was evaluated qualitatively and quantitatively with 3 sequences. A multireader diagnostic optimality evaluation was performed by 16 radiologists. For clinical validation, we evaluated 123 patients who underwent fast stroke MR imaging to assess acute ischemic stroke. The diagnostic confidence level and decision time for large-vessel occlusion were also evaluated.
RESULTS: Median values of overall image quality, noise, sharpness, venous contamination, and SNR for M1, M2, the basilar artery, and posterior cerebral artery are better with synthetic TOF than with time-resolved MRA. However, with respect to real TOF, synthetic TOF presents worse median values of overall image quality, sharpness, vascular conspicuity, and SNR for M3, the basilar artery, and the posterior cerebral artery. During the multireader evaluation, radiologists could not discriminate synthetic TOF images from TOF images. During clinical validation, both readers demonstrated increases in diagnostic confidence levels and decreases in decision time.
CONCLUSIONS: A CycleGAN-based deep learning model was developed to generate synthetic TOF from time-resolved MRA. Synthetic TOF can potentially assist in the detection of large-vessel occlusion in stroke centers using time-resolved MRA.
ABBREVIATIONS:
- AdaLIN
- Adaptive Layer-Instance Normalization
- AIS
- acute ischemic stroke
- BA
- basilar artery
- EMT
- endovascular mechanical thrombectomy
- GAN
- generative adversarial network
- IQR
- interquartile range
- LVO
- large-vessel occlusion
- PCA
- posterior cerebral artery
- PSNR
- peak SNR
- synTOF
- synthetic TOF
- SSIM
- structural similarity index measurement
- TR
- time-resolved
The primary goal of performing MRA in acute ischemic stroke (AIS) is to identify the presence of large-vessel occlusion (LVO), while the secondary objective is to assess the collateral status. Collateral status is the main factor determining how rapidly the penumbral tissue progresses to an irreversible infarct core.1 Recent studies have revealed the significance of collateral imaging in selecting eligible patients for endovascular mechanical thrombectomy (EMT) in the late window, determining the need for transferring the patient to an EMT-capable hospital, predicting prognosis, and determining stroke etiology.1⇓⇓⇓⇓-6 Therefore, the American Heart Association/American Stroke Association 2019 guidelines suggest that incorporating collateral flow status into the clinical decision-making process for eligible candidates may help determine their eligibility for EMT.7
Time-resolved (TR) MRA enables the noninvasive evaluation of collateral status. A set of images is sequentially acquired at multiple time points while the contrast agent passes through the target vessel (Online Supplemental Data).8⇓⇓-11 Several existing techniques for TR-MRA, including time-resolved imaging of contrast kinetics (TRICKS; GE Healthcare) and time-resolved imaging with stochastic trajectories (TWIST; Siemens), use center-weighted keyhole imaging and peripheral undersampling.12 This method is particularly useful for evaluating the collateral or retrograde flow that occurs around stenoses. Nevertheless, compared with TOF-MRA, the inherent SNR penalties associated with undersampling and acceleration techniques further hinder the attainable spatial resolution of TR-MRA, thereby diminishing its usefulness in the assessment of LVO.9,13
The cycle-consistent generative adversarial network (GAN) (CycleGAN; https://github.com/junyanz/CycleGAN), which is a deep learning algorithm, aims to improve image quality through learning between high-quality and low-quality images.14⇓⇓⇓⇓-19 The purpose of this study was to develop a CycleGAN model that generates high-quality synthetic TOF (synTOF) images from low-quality TR-MRA peak arterial phase images and to verify the clinical efficacy of synTOF in diagnosing LVO.
MATERIALS AND METHODS
Patient Selection
This retrospective study was approved by the institutional review board (Korea University Anam Hospital), and the need for informed consent was waived due to its retrospective design. Our study cohort included 1002 patients who underwent TOF-MRA from April 2021 to January 2022. Among them, 397 patients (mean age, 66.94 [SD, 13.79] years, 43.6% women [173/397]) underwent both TOF- and TR-MRA to evaluate their collateral statuses (Fig 1). These 397 patients were divided into a training data set for model development (50.1%, [199/397]; mean age, 66.36 [SD, 14.02] years; 43.7% women [87/199]) and a validation data set for image-quality assessment [49.9%, [198/397]; mean age, 67.58 [SD, 13.56] years; 43.4% women (86/198)]. For clinical validation, 123 patients (mean age, 72.85 [SD, 11.24] years; 41.5% women [51/123]) who underwent fast stroke MR imaging (FLAIR, DWI, perfusion MR imaging, and TR-MRA) to evaluate AIS during the same period were included in this study. Clinical information, such as sex, age, reason for fast stroke MR imaging, location of the AIS, and the LVO site, was collected.
Study design. GRE indicates gradient recalled-echo.
Image Acquisition
MR imaging was performed with two 3T MR imaging scanners (Magnetom Skyra and Magnetom Prisma; Siemens) using a 64-channel head and neck coil. The specific imaging parameters for TOF- and TR-MRA (TWIST) are listed in the Online Supplementary Data. After we obtained a pre-enhancement image, 30 consecutive T1WI 3D data sets were acquired in the coronal plane after automatic injection of 0.2 mL/kg of gadoteridol (ProHance; Bracco), followed by 30 mL of saline. Both sequences were performed in the same session.
Among the dynamic images, the peak arterial phase images were selected to evaluate LVO. MIP images (40 images [anteroposterior 20, lateral 20]) of TOF- and TR-MRA were automatically created under the same conditions by Siemens 3D software (Syngo.via; Siemens Healthineers, Forchheim, Germany).
Model Development
Training Stage.
A schematic diagram of the network architecture and model details are shown in the Online Supplementary Data. This model was developed with paired TR-TOF MIP imaging sets from 199 patients. It includes 2 generators and discriminators, 4 loss functions, and a gradient-weighted class activation map (GradCAM; https://medium.com/the-owl/gradcam-in-pytorch-7b700caa79e5). The basic architecture was based on an unsupervised generative attention network with Adaptive Layer-Instance Normalization (AdaLIN) for image-to-image translation and was customized to improve model performance (https://github.com/taki0112/UGATIT). The encoder is composed of 2 convolution layers (2D, kernel size: 3) with a stride size of 2 for downsampling and 4 residual blocks. The decoder consists of 4 residual blocks and 2 upsampling convolution layers (2D, kernel size: 3) with a stride size of 1. We used instance normalization for the encoder and AdaLIN for the decoder. Hyperparameters used in various training settings are listed in the Online Supplementary Data. GradCAM was incorporated into the foundational CycleGAN model to enable selective attention on specific areas of the image after testing improvement in terms of the structural similarity index measurement (SSIM) in 10 patients randomly selected from the validation data set (Online Supplementary Data). The same 150 training epochs were applied to all cases.
Validation Stage.
The performance of the developed model to generate synTOF (GTOF) was assessed using the peak SNR (PSNR) and SSIM as metrics, defined as follows:20
Image Interpretation
Image-Quality Evaluation (Qualitative and Quantitative Analysis).
Two neuroradiologists (S.E.P and S.-H.Y., with 6 and 11 years of neuroradiologic experience) evaluated the image quality of the validation data set: 594 imaging sets (198 × 3 [TR-MRA, synTOF, and TOF]) from 198 patients (Online Supplementary Data). Each image set consisted of 40 anonymized MIP images and was evaluated through a PACS. Regarding disagreements in the qualitative evaluation, the final result was determined through a consensus meeting with 3 neuroradiologists, including another experienced neuroradiologist (B.K., with 17 years of experience in neuroradiology). The mean value from the 2 readers was used in the quantitative analysis. A 5-point Likert scale was used to evaluate each kind of image-quality parameter (Online Supplementary Data). For quantitative analysis, crucial ROIs were manually located at each vessel, demonstrating the highest signal intensity in the MIP image anterior-posterior view. The SNRs were calculated in the M1, M2, M3, posterior cerebral artery (PCA), and basilar arteries (BAs) (mean signal intensity/SD of the background SI).
Diagnostic Performance for Intracranial Aneurysm and Stenosis Detection.
The same neuroradiologists who performed image-quality evaluation assessed the presence of aneurysms (largest lesion) and intracranial arterial stenoses (most stenotic lesion, exceeding a moderate degree). Another neuroradiologist (B.K., with 17 years of experience in neuroradiology) evaluated both diseases using TOF and VICAST criteria.21
Multireader Image-Optimality Test with 16 Radiologists.
Sixteen radiologists (7 and 9 neuroradiologists with 1–18 years of radiologic experience) evaluated imaging sets from 20 randomly chosen patients in the validation data set. Three image sequences of a patient were randomly arranged and shown simultaneously; then, we asked the following 3 tasks: 1) imaging optimality: select all the imaging sequences suitable for interpretation; 2) image preference: select the best MRA sequence subjectively perceived as optimal for interpretation; and 3) image discrimination: choose 1 MRA sequence that is highly likely to be a genuine TOF.
Assessment of Diagnostic Confidence Level and Decision Time for LVO in Fast Stroke MR Imaging.
For the 123 patients who underwent fast stroke MR imaging, 2 neuroradiologists (B.K.K. and S.E.P., with 12 and 6 years of experience in neuroradiology, respectively) independently graded the diagnostic confidence level for intracranial LVO on the basis of a 5-point Likert scale. The time for the decision was defined as the difference between the time when each MRA image was opened and the time when the decision for LVO was made by the neuroradiologist. In the first session, 2 neuroradiologists evaluated the diagnostic confidence level and decision time using only TR-MRA. After 1 month, both were assessed using TR-MRA and synTOF-MRA simultaneously by the same neuroradiologist.
Statistical Analysis.
The normality of the data distribution was assessed for each parameter using the Kolmogorov–Smirnov test. Patient data are reported as percentage (%), mean (SD), or median (interquartile range, [IQR]). The significance level of P < .05 was used to consider statistical significance. A Bonferroni correction was applied for post hoc analysis, in which the adjusted P value was multiplied by the number of comparisons made. The Friedman test was used to compare >3 non-normally distributed continuous variables, including qualitative and quantitative image-quality analyses and the diagnostic optimality tests. The McNemar-Bowker test was used to compare the distribution of confidence levels between the 2 different imaging sets and the diagnostic performance of 3 imaging sets. The Wilcoxon test was used to compare the nonparametric equivalents of the paired samples, specifically the median values of the diagnostic confidence level and time for decision. Interobserver agreement was assessed on the basis of the analysis of weighted κ values (qualitative analysis) or intraclass correlation coefficients (quantitative analysis).
MedCalc for Windows (Version 20.218; MedCalc Software) and SPSS Statistics software (Version 22.0; IBM) were used for all statistical analyses.
RESULTS
Patient Characteristics
Baseline patient characteristics of the model development and clinical validation data sets are listed in Table 1. In the clinical validation data set, 56 patients (45.5%, 56/123) showed acute ischemic infarction on DWI. Intracranial LVO was present in 25 patients (20.33%).
Clinical characteristics of the patients
Model Development
The median values and IQR of SSIM and PSNR between synTOF and TOF were 0.67 (0.65–0.69) and 15.56 (14.82–16.42) dB, respectively (Online Supplemental Data).
Diagnostic confidence level of TR-alone versus TR and synTOF
Qualitative Assessment
Qualitative analysis results are presented in the Online Supplemental Data. The synTOF showed significantly higher median values on the 5-point Likert scale compared with TR in all the following parameters: overall image quality, noise, sharpness, and venous contamination (P < .001). In comparison with real TOF, synTOF has lower median values in overall image quality, sharpness, and vascular conspicuity (P < .001). The interobserver agreement for all 5 parameters was substantial or excellent (weighted κ > 0.700) (Online Supplemental Data).
Quantitative Assessment
The Online Supplemental Data demonstrate the results of the quantitative analysis. The SNRs of the synTOF in the MCA (M1 and M2 segments) and BA were significantly higher than those in the TR (P < .001) in contrast to the M3 segment (P = 1.000). Compared with TOF, the SNRs of synTOF were significantly lower for the M3, BA, and PCA (P < .001), with no significant differences for M1 (P = 1.000) and M2 (P = .358). The interobserver agreement for all values was excellent (intraclass correlation coefficient > 0.800) (Online Supplemental Data).
Diagnostic Performance for Intracranial Aneurysm and Stenosis Detection
The sensitivity in aneurysm detection for both 4D and synTOF was 50% (Online Supplemental Data). The aneurysm height for all false-negative cases was <2 mm. The sensitivity in stenosis detection for 4D and synTOF was 75% and 86%, respectively. The diagnostic performance of both diseases was significantly lower than that of TOF (P < .05). No significant differences existed between 4D and synTOF (P values; aneurysm = 1.000; stenosis = .063). Representative images are shown in the Online Supplemental Data.
Multireader Imaging-Optimality Evaluation
The results for multireader image-optimality assessment for the 3 questions are presented in the Online Supplemental Data. For the first question, which pertained to imaging optimality for reading, no statistically significant difference was observed between the values for synTOF and real TOF imaging techniques, and both were significantly higher than those obtained using TR (TR versus synTOF versus TOF; median [IQR]; 0.50 [0.00–2.00] versus 20.00 [18.00–20.00] versus 20.00 [18.50–20.00], P = 1.000). With respect to imaging preference, no statistically significant difference was observed between the values for synTOF (median, 4.50 [IQR, 4.00–9.50]) and TOF (median, 15.50 [IQR, 10.00–16.00]) (P = .279). Finally, for the last question, which aimed to select real TOF images, no statistically significant difference was observed between the values for synTOF (median, 10.00 [IQR, 3.50–16.00]) and TOF (median, 9.00 [IQR, 3.00–16.50]) (P = 1.000).
Diagnostic Confidence Level and Decision Time for LVO in Fast Stroke MR Imaging
Table 2 and Fig 2 present the results of the diagnostic confidence level and decision time for LVO. The diagnostic confidence levels were significantly higher for readers 1 and 2 when they determined LVO with both TR and synTOF images than with TR-only (TR and synTOF versus TR-only; reader 1: median, 5.00 [IQR, 4.00–5.00] versus 4.00 [IQR, 4.00–4.00]; reader 2: median, 5.00 [IQR, 4.00–5.00] versus 4.00 [IQR, 3.00–4.00], P < .001). The median values of decision time were significantly shorter in the TR and synTOF reading sessions than in the TR session (TR and synTOF versus TR-only; reader one: 25.00 [IQR, 20.00–42.75] versus 33.00 [IQR, 30.00–38.00], P = .004; reader two: 25.00 [IQR, 22.00–38.00] versus 39.00 [IQR, 30.00–50.00], P < .001).
Diagnostic confidence level of TR-MRA-only versus TR-MRA and synTOF.
Representative TR-MRA, synTOF, and TOF images are shown in Fig 3.
Representative images of TR-MRA, synTOF, and TOF.
DISCUSSION
The major findings of this study are summarized as follows. 1) A high-resolution synTOF generation model based on CycleGAN was developed for the diagnosis of LVO in AIS. 2) In comparison with TR-MRA, synTOF achieves a statistically significant reduction in noise and improvement in sharpness. 3) The model performance is excellent in terms of SNR improvement for M1, M2, BA, and PCA. 4) SynTOF does not accomplish equivalent image quality compared with TOF except for noise, venous contamination, and SNR of M1/M2. 5) SynTOF features limited the performance in detecting intracranial aneurysms and stenoses. 6) The combined use of both synTOF and TR-MRA has the potential to enhance diagnostic confidence and decrease decision time for the diagnosis of LVO in AIS for distal ICA to M2 occlusion.
CycleGAN: Deep Learning Algorithm for Intersequence Image Transfer
Our deep learning model improved the image quality of TR-MRA. This may have been influenced by 2 factors: the use of proper target images (TOF-MRA) and the effectiveness of the CycleGAN algorithm. TOF-MRA is the most commonly used intracranial MRA sequence with high image quality, and neuroradiologists are familiar with its image contrast.22 Therefore, we hypothesized that paired TOF images serve as a valuable guide for enhancing the image quality of TR-MRA. CycleGAN was selected for training because of its recognized capability of intersequence transfer. The GAN was designed to generate new images through simultaneous training of the generator and discriminator networks.19,23 In our study, CycleGAN, a modified version of the GAN with 2 generators and discriminators, was used to prevent mode collapse. Additionally, GradCAM was introduced into the model to drive the focus of the generator and discriminator toward meaningful regions of the images. An effective algorithm and an appropriate target image may have contributed to the favorable performance of the model.
Clinical Application of synTOF in AIS
The use of MRA has 2 objectives in AIS: diagnosis of LVO and assessment of collateral status.7 Considering the first objective and image quality, TOF-MRA was deemed the most suitable method. However, it may not be possible to perform owing to the long scan time. Furthermore, TOF-MRA does not provide adequate information regarding the aortic and proximal cervical arteries, which are technically important for EMT. Single-phase contrast-enhanced MRA offers the advantage of a large FOV and short scan time; however, it is also known to underestimate the collateral status.24,25 TR-MRA is considered the most suitable option for the secondary objective of AIS evaluation, but challenges in attaining maximum efficiency for the primary purpose, owing to limitations in both spatial resolution and SNR, are unavoidable. Our results indicate that the use of synTOF significantly improved the diagnostic confidence level for LVO and reduced the decision time. Thus, in instances in which real TOF acquisition is impeded by time constraints, synTOF images from TR-MRA may serve as valuable additional high-resolution images for the initial evaluation of AIS. However, the diagnostic performance of synTOF in aneurysm and stenosis detection is still limited.
CT versus an MR Imaging–Based Protocol for AIS
Two distinct imaging protocols, namely CT and MR imaging, are used for AIS evaluation. The preference in most stroke centers is CT for AIS evaluation, driven by its predominant advantages over MR imaging, such as rapid acquisition, widespread availability, and cost-effectiveness. A previous study with 36 tertiary hospitals reported that approximately 70% of stroke centers have adopted CT-based protocols.26 Additionally, CTA, including the delayed phase, offers the highest image resolution for evaluating LVO and collateral status. Nevertheless, some centers have explored the use of MR imaging owing to its unique strengths: 1) high accuracy for core infarction, 2) estimation of the onset time in wake-up strokes, 3) effective identification of stroke-mimic conditions, and 4) lack of radiation. In our center, we use a 6-minute MR imaging–based protocol for patients with presumed in-hospital stroke, and the results of our study may benefit MR imaging–based stroke centers. Considering the different goals of deep learning–based image-quality improvement in MR imaging and CT, namely scan time reduction and radiation reduction, respectively, the advantage of CT-based protocols will be boosted by further deep learning–based studies to reduce the radiation and contrast dose while maintaining the image quality.
Limitations
First, the study was a retrospective single-center one. Second, multiobserver imaging evaluation was performed on limited patients, which could have impacted the generalizability of the results. Third, this study had a limitation in that it focused only on a single field strength (3T) and a single vendor sequence (TWIST). Fourth, the number of patients in the clinical validation group with AIS on DWI and LVO on MRA was relatively low, because our hospital only uses an MR imaging–based stroke protocol for inpatient strokes and a CT-based protocol at the emergency stroke center. Additionally, only 3 cases of LVO were included for the posterior circulation. Thus, further studies that include a larger number of patients are needed to determine the usefulness of synTOF in posterior circulation occlusion. Fifth, this model was developed using MIP images rather than raw data. This model is effective for LVO detection. However, raw-data images are essential for a confirmative diagnosis of stenoses and aneurysms. Further studies using raw data and another algorithm should be conducted. Sixth, owing to the widespread use of CT-based protocols in AIS evaluation, the applicability of our research findings is limited. They could be applied only to stroke centers that follow an MR imaging–based protocol. Further studies regarding image-quality improvement through reduction of the radiation dose in CT-based protocols will be needed. Seventh, we did not perform quantitative analysis for vascular sharpness. A reliable quantitative measuring method for sharpness should be developed because resolution enhancement may, essentially, affect image-quality improvement. Finally, DSA was performed on a limited number of patients, precluding a thorough evaluation of the diagnostic accuracy of synTOF- and TR-MRA.
CONCLUSIONS
In this study, we developed a CycleGAN-based deep learning model aimed at generating a synTOF with improved resolution and SNR. Despite the good image quality of TOF, it is often not performed for AIS because of the long scan time. TR-MRA with additional deep learning–based synTOF has the potential to effectively achieve both goals of initial stroke evaluation: accurate and rapid diagnosis of LVO and proper evaluation of collateral circulation.
Footnotes
S-H. You and Y. Cho contributed equally to this study.
This study was supported by the Basic Science Research Program through the National Research Foundation of Korea, funded by the Ministry of Education (grant No. 2021R1I1A1A01050316); a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute, funded by Ministry of Health and Welfare, Republic of Korea (grant No. HR22C1302); and a Korea University grant (grant No. K2225761)
Disclosure forms provided by the authors are available with the full text and PDF of this article at www.ajnr.org.
References
- Received April 12, 2023.
- Accepted after revision October 17, 2023.
- © 2023 by American Journal of Neuroradiology
Jump to section
Related Articles
Cited By...
- No citing articles found.