Deep Learning–Based Detection of Intracranial Aneurysms in 3D TOF-MRA

Dataset

This retrospective study was approved by the Independent Ethics Committee at the RWTH Aachen Faculty of Medicine. The requirement for informed consent was waived. From an internal data base belonging to our department, we incorporated data from all patients with a 3D TOF-MRA examination of at least 1 previously untreated intracranial aneurysm. Images were obtained for clinical purposes between 2015 and 2017. After we removed protected patient information and substituted subject identifiers, examinations were retrieved from the local PACS. The dataset consisted of 85 examinations. Of those, 72 image sets originated from our department. Sixty of these examinations were performed on a 3T scanner (Magnetom Prisma; Siemens; Erlangen, Germany). Twelve examinations were performed on a 1.5 scanner (Magnetom Aera; Siemens).

The following parameters were used for the 3D TOF-MRA: Magnetom Prisma (3T)—TR, 21 ms; TE, 3.42 ms; flip angle, 18°; FOV, 200 mm; section thickness, 0.5 mm; matrix, 348 × 384; acquisition time, 5 minutes 33 seconds; 20-channel head/neck coil; Magnetom Area (1.5T)—TR, 28 ms; TE, 7 ms; flip angle, 25°; FOV, 200 mm; section thickness, 0.5 mm; matrix, 256 × 320; acquisition time, 5 minutes 52 seconds; 20-channel head/neck coil.

Thirteen examinations included in this dataset originated from external departments and were performed on different scanners.

We included all TOF acquisitions with at least 1 previously untreated aneurysm, irrespective of etiology, symptomatology, and configuration (saccular, fusiform, and dissecting). The aneurysms were located in the internal carotid arteries, the anterior cerebral arteries (including the anterior communicating artery), the middle cerebral arteries, or the posterior circulation (including the vertebral, basilar, posterior, cerebral, and posterior communicating arteries). One patient had polycystic kidney disease, while the remainder had incidental findings. Exclusion criteria were previous treatment (coil embolization or surgical clipping) or pronounced motion artifacts, preventing accurate segmentation.

The DeepMedic (Version .6.1; https://biomedia.doc.ic.ac.uk/software/deepmedic/) CNN was used¹⁸ with an application of required preprocessing on the dataset¹⁹: voxel size resampling (0.5 × 0.5 × 0.5 mm³) and intensity normalization to a zero-mean, unit-variance space. To evaluate the impact of preprocessing on the performance of the CNN, we modified our dataset using different BET2 skull-stripping (https://fsl.fmrib.ox.ac.uk/fsl/fslwiki/BET)²⁰ and performing N4 bias correction.²¹

The ground truth segmentation was performed by a neuroradiology resident experienced in cranial diagnostic imaging. On the basis of radiologic reports, anonymized TOF data were critically reviewed, and aneurysms were manually annotated in a voxelwise manner using the manual segmentation tool of ITK-SNAP (www.itksnap.org).²² Intrarater reliability was studied using the Pearson correlation coefficient.

After evaluation of the dataset, we trained DeepMedic and performed inference to segment aneurysms. Remarkably, 2 aneurysms that had been previously overlooked were detected by the CNN in this early stage. Consequently, the dataset was validated by another radiologist who was blinded to the radiology reports. Complete ground truth was evaluated once again and adjusted accordingly.

The dataset needed division into training, test, and validation sets, to run the CNN and assess its performance. The training set was used for learning, which describes the process of fitting the parameters of the network to learn features for discriminating the output classes. The validation set was used during training to reduce overfitting to the training data. This is done by comparing the Dice similarity coefficient (DSC) (a measure indicative of segmentation accuracy) of the training samples with the DSC of the unknown validation samples and adjusting the learning rate of the network. The test set is used for evaluation of the trained model.¹⁸ Training the model took about 20 hours; inference per case was about 50 seconds on a Titan XP GPU (Nvidia, Santa Clara, California).

Five-fold cross-validation was performed. For each split, the whole dataset was randomly divided into the 3 subsets as explained earlier: training set (58 cases, 68%), validation set (10 cases, 12%), and test set (17 cases, 20%).

DeepMedic and Evaluation

Segmentation of the aneurysms was executed with the DeepMedic framework, a CNN for voxelwise classification of medical imaging data after training with 3D patches at multiple scales. DeepMedic was developed and evaluated for the segmentation of brain lesions.²³

The network consists of 2 pathways with 11 layers. Both pathways are identical, but the input of the second pathway is a subsampled version of the first (see the full architecture in Fig 1). Parameters were set as proposed by Kamnitsas et al¹⁸: An initial learning rate of 10⁻³ was used and gradually reduced. For optimization, a Nesterov Momentum of 0.6 was set. For better regularization, drop-out and L1 = 10⁻⁶ and L2 = 10⁻⁴ regularization was performed. To accelerate the convergence, we used Rectified Linear Unit activation functions and batch-normalization as implemented in the DeepMedic framework.²³ We used the proposed DeepMedic hybrid sampling scheme. In this strategy, image segments larger than the neural network's receptive field are given as an input to the network. A training batch is built by extracting segments with 50% probability centered on the foreground or background voxels, facilitating an automatic method for balancing the distribution of training samples regarding the size of the desired class in the segment and therefore preventing class imbalance by adjusting to the true distribution of background and aneurysm voxels.¹⁸ With a probability of 50%, the training images were mirrored on the coronal axis to increase the diversity of the training set.

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

Flowchart of the pipeline. A, Preprocessing is performed with 4 different models. The dataset is split into test, training, and validation sets. B, Inference is performed with the convolutional neural network DeepMedic with a 2-pathway architecture. The number of feature maps and their size is depicted as number × size. The + depicts the addition of the 2 preceding layers, which adds an additional nonlinearity and reduces the number of weights.¹⁸ The diagram is based on the depiction in the DeepMedic documentation. (Modified from Kamnitsas K, Ledig C, Newcombe VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 2017;36:61–78 under CC-BY4 license).³² C, Thresholding is applied to the resulting segmentation and evaluated with different metrics. L_N indicates layers in the normal resolution pathway, L_L indicates layers in the low resolution pathway.

We used the EvaluateSegmentation Tool (https://github.com/Visceral-Project/EvaluateSegmentation)²⁴ to analyze the segmentation results by determining Hausdorff distances and the DSC. For methodologic reasons, each segmented voxel or connected component of voxels in the output binary segmentation was considered a positive detection. Each positive detection that corresponded to an aneurysm in ground truth was considered a true-positive finding, while each positive detection that did not correspond to an aneurysm in ground truth was considered a false-positive finding. In preliminary studies, this approach led to a very high rate of false-positive detections. Because we observed that compared with true-positive detections, false-positives tended to be rather small, we further examined whether the integration of a detection threshold as a postprocessing step, removing connected components smaller than a given volume, would improve our results. On the basis of the composition of our dataset, detection thresholds of 5, 6, and 7 mm³ were studied (Fig 1). To further reduce the number of false-positives, we fine-tuned the network using a modified training strategy in which 90% of the input samples corresponded to background class; and 10%, to aneurysm class, reflecting a more realistic distribution of aneurysms. The learning rate was lowered to 10⁻⁴ and the pretrained weights of the last 3 layers were changed while the training weights of the other layers were kept constant. To study the reliability of true-positive detections and the capability of the system in predicting aneurysm size, we compared the volume segmented by the algorithm with the manually examined volume of the ground truth.

To assess the impact of preprocessing, we evaluated 4 models (A–D). In model A, only the necessary steps to obtain reasonable results from DeepMedic, resampling to isotropic voxel size and intensity normalization, were performed. Additional skull-stripping is advised in the DeepMedic documentation.¹⁸ We used the well-established BET2 skull-stripping method. Skull-stripping in model B was performed with a fixed fractional intensity threshold of 0.2. In model C, the parameter was adjusted manually in each case to receive an optimal brain outline, without nonbrain structures such as skull or parts of the ocular muscles and nerves. For model D, we used the skull-stripping masks from model C and performed an additional N4 bias correction²⁵ to evaluate whether low-frequency intensity inhomogeneities in the acquisitions would have an impact on the performance of the algorithm (Fig 1). In this work, each model is depicted as a preprocessing model identifier (A–D), followed by the detection threshold (0, 5, 6, 7).

Full preprocessing per case took about 5 minutes on a Corei7–8700K CPU (Intel, Santa Clara, California). Individual creation of a skull-stripping mask was performed by an experienced user and took about 8 minutes for each sample.

Statistical analysis was performed using SPSS software, Version 25.0 (Released 2017; IBM Armonk, New York). We used the Shapiro-Wilk test to test for normality. Significance values of normality tests are only reported for cases in which the normality assumption was violated. A Kruskal-Wallis test was used for the split-validation of maximum diameters.

Comparisons among the Models

We hypothesized the 4 different levels of preprocessing to each be improvements over the previous version. Therefore, sensitivity values of each preprocessing model were compared only with those of its closest neighbor by testing for differences in the proportions of hits and misses, using McNemar tests. These tests were chosen over χ² tests because the values obtained from each model were not independent of one another. Comparing each model with its closest neighbor yielded 3 comparisons (A0 versus B0, B0 versus C0, C0 versus D0); thus, significance levels were corrected for 3 comparisons using a Bonferroni correction.

False-positives per case (FPs/case) were compared for each preprocessing model using a Friedman test. Post hoc tests were run using Wilcoxon signed rank tests for each closest neighbor.

DSCs of each preprocessing model were compared using Friedman tests. Post hoc tests were run using Wilcoxon signed rank tests for each closest neighbor. Missing values, caused by the inability of the evaluation tool to analyze volumes with no segmented voxels, were set to zero.

Hausdorff distances of each preprocessing model were analyzed using a linear mixed model, which included a random subject factor, and “model” as the sole fixed dependent variable. This linear mixed model was chosen over a repeated-measures ANOVA because the linear mixed model can analyze missing values better; unlike DSCs, a Hausdorff distance of zero would not accurately describe the inability of the tool to analyze a volume with no segmented voxels.

Comparisons within the Models

We hypothesized that each of the postprocessing models reduces the number of false-positives sequentially. Thus, sensitivity values for each detection threshold were compared with those of the closest neighbor within each model by testing for a difference in the proportions of hits and misses using McNemar tests. This yielded 3 comparisons per preprocessing model (0 versus 5, five versus 6, and 6 versus 7).

FPs/case were compared for each detection threshold using a Friedman test. Post hoc tests were run using Wilcoxon signed rank tests comparing each closest neighbor.

Size and Location

Increased aneurysm size embodies an increased rupture risk.⁴ However, consented classifications of aneurysms based on aneurysm size are missing. To study the impact of aneurysm size on the detection rate, we classified aneurysms on the basis of maximum diameter as follows: In the literature, aneurysms with a maximum diameter of ≤3 mm are generally considered tiny.²⁶ For simplification, we termed these findings small aneurysms. A distinct increased risk of rupture was identified for aneurysms with a diameter of >7 mm.⁶ We therefore defined aneurysms of >3 but ≤7 mm as medium, and those of >7 mm as large. Additionally, aneurysms were categorized on the basis of their location.

Sensitivity values of these categories were compared for both categorizations using Fisher exact tests rather than χ² tests because the cases numbered below 5 for certain cells. Spearman rank correlation coefficients were calculated between ground truth and predicted volumes because the normality assumption was violated in all samples.