Individual Detection of Patients with Parkinson Disease using Support Vector Machine Analysis of Diffusion Tensor Imaging Data: Initial Results

Group-Level TBSS Analysis

The first part of the analysis was a group-level analysis of white matter changes in subjects with suspected PD.

Most previous advanced MR neuroimaging studies in the domain of PD compared subjects with PD with healthy controls, with the objective of identifying disease-related alterations in brain morphometry. For example, one of the rare DTI studies in PD demonstrated a lower FA in the substantia nigra in PD compared with healthy controls, which inversely correlated with the clinical severity of PD.² Two studies used manually defined ROIs in the substantia nigra showing diminished FA in PD versus healthy controls.^2,28 Another study used multiple ROIs in 10 patients with PD without dementia and 10 healthy controls, showing decrease in FA and increase in MD in the genu of the corpus callosum and in the superior longitudinal fasciculus.⁵ Yet other studies found differences in DTI FA in patients with PD with olfactory impairment,²⁹ patients with PD and depression,³⁰ or between patients with PD and familial essential tremor⁴ or corticobasal syndrome.³¹ Another recent study assessed different MR imaging parameters including R2*-, R2-, and R1-mapping, magnetization transfer, and DTI in 31 patients with various forms of Parkinsonism and found that manually defined ROIs of DTI are most useful for identifying PSP, yet less useful for PD.³² Most interesting, a recent study assessed white matter in 36 subjects with PD and 23 controls on the basis of another technique, notably magnetization transfer imaging.³³ PD-related changes in WM were more sensitive than gray matter volume or attenuation derived from T1-weighted images.

All of these studies have in common that they included very selected control groups.

In contrast to these studies, the aim of the present investigation was to use DTI as a marker to detect individual patients with PD in a group of subjects with suspected PD. We, therefore, deliberately included consecutive nonselected patients with various forms of Parkinsonism. We observed, in our sample, alterations in FA, a measure of axonal integrity,³⁴ as well as RD and MD in a bilateral right-dominant frontal network. The observed increase in FA in our study is in agreement in principle with the observed increase in FA in de novo patients with PD versus healthy controls in a recent combined T1-weighted and DTI study,³ while another study in 12 subjects with nondemented PD and 13 controls demonstrated a decrease in FA bilaterally in the frontal lobes.⁶ Differences in disease duration, severity, and characteristics might explain this discrepancy. Moreover, the increase in FA in PD versus Other subjects in our study is probably due to a decrease in FA in the Other group, rather than due to an increase in FA in the PD group. Due to the heterogeneous constitution of our control group, the results of our group-level comparison should be interpreted with caution and are presented mainly to visualize the presence of detectable DTI changes between subjects with PD versus Other, as a basis for the later individual-level pattern-recognition analysis.

Individual-Level SVM Classification Analysis

To obtain individual discrimination of subjects with PD, we adopted a complex methodology including a chain of TBSS preprocessing of DTI FA data, feature selection of the most discriminative voxels, and subsequent SVM classification.^15,35 The classification accuracy of approximately 97% across the 10 repetitions of the 10-fold cross-validation implies that on average, only 1 subject was incorrectly classified.

Note that SVM⁷ analyses for individual classification are fundamentally different from the group-level voxelwise analyses discussed above. Such voxelwise analyses are univariate tests, which separately analyze each included voxel between 2 (or more) groups. Given the multiple tests, in our dataset of approximately 150,000 voxels, it is necessary to implement, as a second step, a correction for multiple comparisons. In contrast, individual-level SVM analyses are multivariate tools that originate from a field called “machine learning” or multivoxel pattern analysis, a branch of artificial intelligence. The aim is to identify patterns that allow the discrimination of individual subjects. There is only 1 resulting parameter per subject; hence, there is no need for corrections for multiple comparisons. For a more detailed discussion of SVM classifiers, see a recent review by Haller et al.⁸

There are only a few previous applications of SVM classification in the domain of PD. Most of these studies applied SVM classifiers to behavioral data of gait analysis,³⁶ fine-motor force tracking,³⁷ and analysis of wearable accelerometer sensors³⁸ or joint movement,³⁹ and even the recorded voice.⁴⁰

The only previous SVM application to MR imaging data in the domain of PD analyzed VBM-preprocessed gray matter in 21 patients with PD, 11 with progressive multisystem atrophy and 10 with PSP, and 22 healthy controls.⁴¹ The best classification accuracy up to 96.8% was obtained for PSP versus PD, while the accuracy was 71.9% for MSA versus PD. However, patients with PD could not be discriminated from controls. These classification accuracies are consistent with the clinical neuroradiologic experience because PSP has the most pronounced visible alterations in brain MR imaging with atrophy of the mesencephalon referred to as the “penguin” or “hummingbird” sign,⁴² while changes in progressive multisystem atrophy are already less pronounced and, as discussed above, PD-associated changes are very subtle.

Neuroradiologic research has been dominated for decades by group studies. The patient groups for such studies should ideally be preselected to have homogeneous and matched groups with typical disease. This is fundamentally different for individual-level classification studies because the preselection of cases might systematically confound the performance of a classifier in a typical clinical setting consisting of unselected and consecutive cases. The preselection of an “artificially” homogeneous atypical Parkinsonism group would train the classifier to detect regions that best discriminate idiopathic PD versus the group of preselected atypical Parkinsonism, yet these regions are not necessarily those that best discriminate between idiopathic PD versus unselected and consecutive patients with suspected PD in a clinical setting. Note the prevalence of these diseases is very different. According to Medscape (http://emedicine.medscape.com), the prevalence in the United States of, for example, progressive multisystem atrophy is 1.9–4.9 cases per 100,000 population (http://emedicine.medscape.com/article/1154583-overview#a0199), yet around 120 (range, 18–328) in PD (http://emedicine.medscape.com/article/1154583-overview).

With respect to a potential clinical application of a classifier, the study populations should ideally match the prevalence of the diseases: A very high accuracy of detection of a very rare disease may overestimate, while the incorrect classification of a rare disease may underestimate, the classification accuracy in “real-world” data. Moreover, preselection of patients typically results in the inclusion of “classic” cases, which may have stronger disease-related alterations, while “real-world” data may contain fewer “classic” cases. In other words, the preselection of patients with specific diseases might represent a systematic confound with respect to the performance of a classifier in “real-world” data, and we consequently included consecutive unselected patients from our institution.

Most previous applications of the SVM classification to brain MR imaging data were performed on gray matter VBM data in the domain of Alzheimer disease⁴³ and mild cognitive impairment,^44⇓–46 with classification accuracies between 75% and 85%. One recent investigation also assessed DTI data in the domain of mild cognitive impairment with accuracies over 95%,¹⁵ implementing equivalent methodology to that used in the current investigation. This suggests that DTI might be a very sensitive brain MR imaging parameter. Consistent with this observation, several recent group-level investigations in various neurodegenerative disorders demonstrated that white matter DTI TBSS analysis is more sensitive than gray matter VBM.^{6,9⇓⇓–12} This does not necessarily imply that WM pathology is more pronounced at a histopathologic level; it might simply be a methodologic difference in data acquisition and preprocessing sensitivity favoring white matter DTI TBSS over gray matter T1-weighted-based VBM. Note that FA is a directly assessed absolute parameter between 0 and 1. In contrast, VBM is based on relative 3D T1-weighted values. VBM consequently requires preprocessing that segments the brain into different compartments to provide an indirect probabilistic gray matter probability per voxel. Additionally, at least the major tracts of the white matter skeleton (TBSS) are generally more linear and have less interindividual variation compared with the much more complex superficial gyral and sulcal folding pattern (VBM), which might imply a different quality of the spatial normalization of the data.

Limitations

The major limitation of the present investigation is the relatively small number of cases, which may affect the results of the SVM analysis, and we propose the current investigation as preliminary data. In fact, the very high accuracy rates of individual classification exceeded our expectations. Both training and testing were performed on the same dataset. A simple split of the data into 2 halves, by using one-half for training and the other half for testing is problematic in small sample sizes such as in the current study because this decreases the number of instances to train the classifier, and the classification accuracy might depend on the division of cases. The reported values were obtained by a well-established 10-fold cross-validation in which 9 parts were used for training and the remaining part was used for testing the classifier. This procedure was repeated 10 times, so that each dataset was used once for testing. This approach increases the sample size for the training of the classifier and at the same time reduces variation of the classification results due to division of cases.

Even though this cross-validation approach is a standard method in the field of machine learning/multivoxel pattern analysis and appropriate for the number of subjects involved in our study, the present results are too optimistic, related to some degree of overfitting of the data. Future validation of the present findings is warranted in a larger and independent sample, which ideally should be acquired on different MR scanners. The consecutive and unselected composition of the Other group is another limitation, yet the rationale behind this selection is discussed in detail above. Additionally, the nonlinear (radial basis function-kernel) SVM does not provide an easy-to-interpret weight vector to identify the most discriminative brain areas. Another limitation is the retrospective nature of the present study.