Evaluation of Lower-Dose Spiral Head CT for Detection of Intracranial Findings Causing Neurologic Deficits

Patients and Cases

The primary diagnostic task for this study was to identify imaging findings that may require further treatment or evaluation or that may potentially explain patient signs or symptoms in patients with suspected acute neurologic deficit. After approval by the institutional review board of Mayo Clinic, we archived CT image and projection data of patients who underwent clinically indicated spiral head CT examinations for suspected acute neurologic deficit and who provided consent for the use of medical records for research purposes. CT image and projection data were collected between August 20, 2013, and May 3, 2014. Archived CT images were then reviewed by a board-certified reference neuroradiologist (D.R.D., with 19 years of experience as an attending neuroradiologist) to ensure that inclusion and exclusion criteria were met for this retrospective, case-control study.

All patients underwent CT for suspected acute neurologic deficit. Inclusion criteria were different for positive and negative cases but required reference criteria to be met. Positive cases were additionally required to have sufficient clinical evidence to meet reference standard criteria for at least 1 of 4 target conditions causing acute neurologic deficit: 1) infarction (acute, subacute, chronic, or indeterminate age), 2) intra-axial hemorrhage (eg, contusion), 3) mass, or 4) extra-axial hemorrhage (eg, subdural, subarachnoid, epidural, intraventricular). Reference standard criteria for accepting positive cases into the study required confirmation of an imaging abnormality on the CT examination by the reference radiologist, in addition to the following: 1) clinical physician confirmation of neurologic deficits at physical examination corresponding to abnormal imaging findings on the index CT examination, 2) progression or confirmation of imaging findings on another imaging examination (eg, subsequent MR imaging, CT perfusion), or 3) confirmation of imaging findings at an operation. The reference neuroradiologist, unblinded to all clinical information and all subsequent imaging and surgical reports, then marked CT images for all CT findings relating to the target diagnosis that were present within the imaged volume that met inclusion criteria using a specially designed computer workstation, tightly circumscribing each CT imaging finding relating to each diagnosis and documenting correlative reference information using standard menus. An ROI was also placed within white or gray matter (as appropriate) to reflect the background CT number in which the target lesion was located.

Negative cases were required to have sufficient evidence that a suspected acute neurologic deficit was not present. Reference standard criteria for accepting negative cases into the study included both the clinical neuroradiologist interpreting the head CT at the time of imaging as well as the unblinded reference neuroradiologist indicating that no imaging findings associated with the 4 target conditions were present, and lack of focal neurologic findings on physical examination by the clinical attending neurologist. Imaging findings of small vessel ischemic change associated with aging (leukoaraiosis) were noted in all included CT exams whether or not one of the four target diagnoses were present.

To determine the lowest radiation dose at which observer performance was noninferior to routine dose, we constructed the study cohort to display visually challenging imaging findings that might affect a radiologist's ability to detect the target diagnoses at different radiation dose levels. Obvious CT findings such as a large intracranial hemorrhage can be detected at even the 10%–20% dose levels,¹² and inclusion of such obvious cases does not help discriminate the diagnostic differences among dose levels. The nonreader reference neuroradiologist visually evaluated 857 CT examinations (620 negative and 237 positive) that met all reference and inclusion criteria and graded the conspicuity of abnormal imaging findings associated with the target diagnoses along a 4-point scale: 1) minimally evident (eg, minimal obscuration of the lentiform nucleus and insular cortex in new infarcts that might be easily missed), 2) subtle (eg, more definite acute infarct or small metastases on noncontrast spiral head CT), 3) distinct abnormality with well-defined borders (eg, small chronic infarct, small intraparenchymal hematoma), or 4) obvious finding (eg, chronic territorial infarct, large acute intraparenchymal hematoma, or diffuse subarachnoid hemorrhage). Thus, cases with obvious imaging findings that would have no discriminatory value in selecting the appropriate radiation dose level would not be included. On the basis of this subjective conspicuity score, visually challenging positive cases that met reference criteria were selected from among the collected positive cases meeting inclusion criteria (Fig 1).

• Download figure
• Open in new tab
• Download powerpoint

Fig 1.

Study schema.

The target population for this study was constructed as previously described, with a population of 83 patients, approximately half of whom would have 1 of the 4 target lesions causing acute neurologic deficit.¹²

Image Acquisition and Reconstruction

Unenhanced spiral head CT examinations were acquired using nearly identical CT platforms (Definition FLASH or Definition AS+; Siemens, Erlangen, Germany) using a single x-ray tube, a detector configuration of 64 × 0.6 mm with a z-flying focal spot yielding 128 detector rows, 1-second tube rotation time, 120 kV(peak), and either 250 effective mAs (eff. mAs) or 340 eff. mAs. The higher tube current was obtained if the examination was part of a trauma scan in our emergency department. Routine-dose CT images were reconstructed using a J40 head kernel using iterative reconstruction with a strength of 2 (sinogram-affirmed iterative reconstruction [SAFIRE]; Siemens), with 5-mm-thick images reconstructed every 5 mm in the axial plane and 2-mm-thick images reconstructed every 2 mm in the coronal plane. CT images corresponding to lower-dose levels were created by inserting image noise in CT projection data using a highly accurate and validated noise-insertion tool.^5,6,13,14 For projection data obtained using 120 kV and 340 eff. mAs, noise was inserted to obtain images corresponding to 250 eff. mAs so that all patient “routine” dose examinations corresponded to the same dose level. On the basis of the prior results, CT projection data dose levels corresponding to 100, 50, and 25 eff. mAs were then also created using iterative reconstruction at these dose levels, and additionally with FBP at 50, 100, and 200 eff. mAs (Table 1). Noise-inserted CT projection data for each case were subsequently loaded back onto the CT system to reconstruct corresponding axial and coronal images with the appropriate reconstruction kernel.

Image Evaluation by Neuroradiologists

Three neuroradiologists with 18, 18, and 5 years of experience as clinical neuroradiologists at our institution were selected as blinded radiologist readers. Because of the unique features of head CT (complicated anatomy, variety of normal aging processes not representing pathology), a standardized reader training manual that defined pathologies to be detected and instructions for reporting reader confidence scores (with anchors) was developed and reviewed by each participating neuroradiologist (On-line Appendix). Confidence scores ranged from 0 (indicating certainty that the circumscribed finding is not one of the target lesions) to 100 (indicating the highest degree of certainty that can be achieved with CT for one of the target findings).^15,16 Neuroradiologists were instructed not to mark frequently seen aging processes such as small-vessel ischemic change (leukoaraiosis), benign intraparenchymal calcification, chronic lacunar infarctions, or arachnoid cysts. Formal one-on-one reader training with the principal investigator was completed, with each reader interpreting 20 training cases selected to match the case mix, pathologies, and dose-reconstruction configurations in the subsequent reader study, discussing reader confidence ratings and any questions.¹⁷

Readers evaluated routine-dose–reconstruction and 6 lower-dose–reconstruction configurations using a specialized computer workstation viewing images using at least 2 window settings (80/40 and 33/40) in multiple sessions. Once a CT finding corresponding to a target diagnosis was identified, readers were instructed to tightly circumscribe all imaging abnormalities corresponding to one of the target diagnoses using a spline tool. Readers rated their level of confidence that one of the target diagnoses was present. Subsequently, readers answered 4 image-quality questions related to overall quality, image sharpness, image noise, and noise texture based on the modified European Quality Criteria, with overall image quality rated along a 5-point scale (1, nondiagnostic due to excessive noise or artifacts; 2, diagnosis questionable due to excessive noise or artifacts, moderate decrease in confidence; 3, diagnostic with moderate-but-acceptable noise or artifacts; 4, mild noise, no change in confidence; and 5, routine diagnostic image quality).^14,18 Examinations were interpreted in random order, with only 1 dose-reconstruction configuration per patient interpreted during each session.

Statistical Analysis

The sample size for this study was determined as a part of a 3-stage study design with the objective in this stage to screen and prioritize imaging strategies for evaluation in a large, future 10-reader, multicase study (stage 3). Stage 1, which consisted of 43 independent cases, has been previously published.¹² The original sample size calculations determined that 83 cases were needed for this second stage of the study.

Matching of colocalized reference and reader detections was performed by the reference neuroradiologist (Fig 2). Comparison of reader performance between routine-dose head CT (at 250 eff. mAs) using a head IR kernel (J40 strength 2) with lower-dose reconstruction configurations was performed using reader agreement rules and jackknife alternative free-response receiver operating characteristic figure of merit (JAFROC FOM) noninferiority analysis. Reader agreement rules compared reader localizations on the routine dose with the lower-dose configurations. For positive cases (i.e., one of the four target diagnoses was present), readers had to localize all target lesions, which were identified by 2 of 3 neuroradiologist readers at routine-dose configurations (denoted as “essential lesions”). For negative cases without imaging findings corresponding to a focal neurologic deficit, no false-positive localizations could be made by ≥2 of the neuroradiologist readers. Preset criteria for prioritization of a lower-dose configuration were agreement with routine dose interpretation in 71 of the 83 cases (86% of examinations).⁸

• Download figure
• Open in new tab
• Download powerpoint

Fig 2.

Small right thalamic hemorrhage (white arrow) shown on routine-dose CT image (250-eff. mAs IR) along with lower-dose configurations. The small left inset shows reference neuroradiologist markings of the target lesion (green circle). This CT examination was performed after trauma, with hemorrhage confirmed surgically, and the final diagnosis was recorded as right thalamic hemorrhage consistent with shear injury.

JAFROC FOM analysis used the reader confidence scores input by the readers as well as their circumscribed imaging findings. A full description of JAFROC FOM for a mixed population as in this study has been provided previously.^19,20 When >1 imaging finding was present in positive cases, findings were weighted according to the reciprocal of the number of findings. FOMs were calculated for every dose level and every reader. The contrasts (comparisons) of FOMs were estimated using the Hillis improvement (Dorfman, Berbaum, and Metz method under the modeling assumption of fixed readers, random cases using the Rjafroc package, Version 1.0.1 [https://cran.r-project.org/]). Noninferiority of lower-dose configurations was represented by calculating the estimated difference between routine and lower-dose configurations, with the limit of noninferiority set at −0.10, a value determined a priori on the basis of investigator consensus. This limit means that when the lower limit of the 95% confidence interval is greater than −0.10, noninferiority with observer performance at a routine dose is achieved.

In addition to these analyses, typical measures of diagnostic accuracy such as per-patient sensitivity and specificity and per-lesion sensitivity were performed. For the purposes of this calculation, a cutoff reader confidence of 10 (of 100) for one of the target findings was used. For per-patient specificity, there could be no reader circumscriptions with a confidence level of >10 in negative cases. For these measures of diagnostic performance, generalized estimating equations (GEEs) were used to estimate the pooled estimate across the 3 readers for each imaging strategy.

For the image quality ratings (eg, overall impression of diagnostic image quality), a summary score was computed for each of the datasets. The summary score was the mean of the 3 readers. Tests for differences in image quality across dose and reconstruction were facilitated by a mixed model consisting of a random patient factor to account for the correlation of image qualities across the doses and assumed Gaussian errors. Post hoc comparisons of the quality summary score were considered descriptive and were not adjusted for multiple testing across doses. We evaluated the effect of certain CT lesion characteristics such as size, CT number difference, and contrast-to-noise ratio compared with adjacent normal-appearing brain parenchyma on observer performance using Spearman rank coefficients comparing these parameters with mean reader confidence for correctly detected lesions. For false-negative examinations, a confidence score of zero was assigned.