You are here
Neurologists׳ accuracy in predicting cognitive impairment in multiple sclerosis
Multiple Sclerosis and Related Disorders, Vol. 4, Issue 4, July 2015, pages 291 – 295
Cognitive impairment affects approximately 40–70% of MS patients. As management of MS typically begins with, and is co-ordinated by neurologists, they are often the first to raise concerns about a patient׳s cognitive functioning. However, it is not known how accurate the neurological examination is in identifying cognitive impairment. To this end, we conducted a retrospective chart review of 97 MS patients referred by neurologists for neuropsychological assessment based on suspected cognitive impairment. Patients were classified as globally-impaired or intact according to failure on 2 or more of 11 cognitive indices comprising the MACFIMS, a recommended neuropsychological battery for MS. Neurologists׳ accuracy was not significantly different from chance, Χ2=1.25, p=0.26, with 44.3% of patients with suspected cognitive impairment showing global impairment on objective testing. Cognitively intact patients when compared to those who were impaired had higher levels of education and were less likely to have mood disturbances. These findings indicate the clinical interview and standard neurological examination are not sufficiently sensitive to detect cognitive impairment in MS, and suggest the need for a brief, accurate cognitive screen to complement routine clinical evaluation.
- A chart review examined the detection of cognitive deficits in multiple sclerosis.
- Routine neurological exam did not predict deficits on neuropsychological assessment.
- Neurologists were at chance in classifying patients as cognitively impaired.
- Patients with cognitive impairment had less education and more mood disturbances.
- Routine visits for MS patients should include some neuropsychological testing.
Keywords: Cognitive impairment, Neuropsychological assessment, Multiple sclerosis, Neurological exam.
Cognitive impairment affects roughly 40–70% of MS patients ( Benedict and Zivadinov, 2011 ). Consequently, accurate measurement of cognitive functioning is an important aspect of managing overall care. Formal neuropsychological assessment is the standard for determining cognitive impairment and measuring cognitive changes, and provides crucial information that influences a patient׳s functional independence (Benedict and Zivadinov, 2011 and Langdon, 2010), work capacity ( Benedict et al., 2005 ), and overall quality of life ( Glanz et al., 2010 ). Assessing neuropsychological performance over time also informs clinicians with respect to monitoring treatment ( Benedict, 2005 ). However, access to such services is often limited to neurologists working in university-affiliated hospitals or within the private sector by cost.
Despite the importance of identifying cognitive changes in MS, little is known about the accuracy of identifying cognitive impairment based on clinical judgment. As neurologists are typically the first line in managing MS patients, determining the accuracy of routine history taking and neurological evaluation in raising concerns about cognitive impairment is of immediate clinical relevance. Thirty years back, a study exploring the accuracy of neurologists in predicting cognitive impairment revealed that approximately half of patients deemed intact turned out to be impaired based on neuropsychological inquiry ( Peyser et al., 1980 ). Given the increase in research on cognition in MS over the subsequent three decades, improved awareness of cognitive deficits in MS may have translated into improved accuracy in identifying cognitive impairment in these patients. To this end, we sought to estimate the current accuracy of neurologists׳ ability to detect cognitive impairment, by conducting a retrospective chart review of MS patients with suspected cognitive impairment referred for neuropsychological testing.
2. Materials and methods
We reviewed the charts of patients with a confirmed diagnosis of MS ( Mcdonald et al., 2001 ) referred for cognitive testing to a Neuropsychiatry service between January 2010 and January 2014. Referrals were made by neurologists based on suspected cognitive impairment. Exclusion criteria for the purpose of the present inquiry were a history of co-morbid neurological disorders that could affect cognition (i.e. stroke, traumatic brain injury), developmental delay and when English as a second language was thought to have influenced the overall cognitive results. In addition, patients who were missing neuropsychological test scores or other clinical variables were not included in the study, resulting in a total of 97 patients included for analysis.
Demographic information, including, age, sex, years of education, and employment status at the time of neuropsychological assessment were extracted from patient files. In addition, MS-related variables (disease course, disease duration, Expanded Disability Status Scale scores, and use of disease-modifying drugs) were also recorded.
The neuropsychological assessment included the Minimal Assessment of Cognitive Functioning in Multiple Sclerosis (MACFIMS) ( Benedict et al., 2002 ), a cognitive battery covering the most common cognitive deficits observed in MS: specifically, tests of Information processing speed (Symbol Digit Modalities Test [SDMT]; Paced Auditory Serial Addition Task [PASAT]) (Gronwall, 1977 and Smith, 1982), verbal memory (California Verbal Learning Test [CVLT-II]) ( Delis et al., 2000 ), visual memory (Brief Visuospatial Memory Test Revised [BVMT-R]) ( Benedict, 1997 ), verbal fluency (Control Oral Word Association Test [COWAT]) ( Benton et al., 1994 ), visual perception (Judgment of Line Orientation [JLO]) ( Benton et al., 1994 ), and executive functioning (Delis–Kaplan Executive Functioning System – Card Sorting Test [CST]) ( Delis et al., 2001 ). Patients were classified as having global cognitive impairment based on the presence of two or more test scores falling 1.5 standard deviations or below the age- and education-corrected norms where available. Patients were also given the Hospital Anxiety and Depression Scale ( Zigmond and Snaith, 1983 ). All assessments were conducted by a post-doctoral fellow in neuropsychology (K.R.) or an experienced psychometrist, both under the supervision of a licensed neuropsychologist (P.S.).
We used the Chi-squared statistic to determine neurologists׳ accuracy in predicting cognitive impairment. Specifically, we calculated the frequency of patients with suspected cognitive impairment who were cognitively-intact or -impaired on objective neuropsychological tests, and compared those frequencies to chance performance (50%). This allowed us to determine whether neurologists׳ ratings as a whole were significantly different from chance. Subsequently, we also compared demographic and clinical variables between cognitively-impaired and intact MS groups using independent samples t-tests. These comparisons provided additional information regarding factors that differ between patients with and without cognitive impairment.
All research protocols were approved by the research ethics board at Sunnybrook Health Sciences Center prior to the study.
Of the 97 MS patients whose data were analyzed, 44.3% (43/97) had confirmed global cognitive impairment. Neurologists׳ accuracy in predicting cognitive impairment in the whole MS patient sample was not significantly different from chance, Χ2=1.25, p=0.26. We also conducted separate analyses to determine whether predictive accuracy differed across disease course. Patients were grouped into those with relapsing-remitting or progressive (primary or secondary) disease courses. Accuracy did not differ as a function of disease course, with neurologists accurately identifying cognitive impairment in 44.6% of patients with relapsing-remitting MS (37/83; Χ2=0.98, p=0.32), and in 42.9% of patients with primary or secondary progressive MS (6/14; Χ2=0.29, p=0.59).
Rates of cognitive impairment for each neuropsychological test are listed in Fig. 1 . The most common cognitive deficits appeared on processing speed tests, with 33–35% of patients falling in the impaired range. Visual memory was also commonly affected, with impairments occurring in 32% of patients.
When compared to cognitively intact MS patients, those with cognitive impairment had lower levels of education (t=2.60, p=0.011), higher levels of depression and anxiety (t=−3.25, p=0.002; t=2.20, p=0.031) and were less likely to be employed at the time of assessment (Χ2=4.11, p=0.04). There were no differences between groups in terms of EDSS scores (t=1.53, p=0.13) ( Table 1 ).
|Mean (SD)||Mean (SD)|
|Age||42.87 (12.57)||40.86 (10.76)||0.83||0.41|
|Percent female # (%)||38 (70.4%)||30 (69.8%)||0.95||1.00|
|Education (years)||16.53 (3.02)||14.86 (3.29)||2.60||0.01|
|MS disease course # (%)|
|Relapsing-remitting||45 (55.6%)||36 (44.4%)||–||–|
|Benign||1 (50%)||1 (50%)||–||–|
|Secondary progressive||3 (37.5%)||5 (62.5%)||–||–|
|Primary progressive||5 (83.3%)||1 (16.7%)||–||–|
|Illness duration (years)||9.14 (7.53)||9.49 (7.22)||−0.23||0.82|
|EDSS a||2.00 (1.97)||2.71 (2.13)||−1.53||0.13|
|Employment status # (%)||30 (55.6%)||15 (33.3%)||4.11||0.04|
|HADS Anxiety subscale||9.09 (3.82)||10.93 (4.18)||−2.20||0.03|
|HADS Depression subscale||5.91 (3.47)||8.50 (4.22)||−3.25||0.00|
a Scores not available for all patients. Statistics based on n=46 (intact) and n=34 (impaired) MS patients.
MS=multiple sclerosis; EDSS=Expanded Disability Status Scale; HADS=Hospital Anxiety and Depression Scale.
Our results showed that neurologists׳ predictions of cognitive impairment based on a typical clinical visit are not significantly different from chance, confirming earlier evidence that the routine neurological assessment lacks sensitivity in identifying cognitive impairment (Peyser et al, 1980 and Benedict, 2005). Thirty years back, Peyser et al. compared neurologists׳ impressions of cognitive impairment against a standard neuropsychological test of conceptual reasoning. Of the 52 patients reviewed, 6/7 patients identified as cognitively impaired indeed demonstrated objective cognitive impairment: however, of the remaining 45 patients judged to be intact, 22 were in fact cognitively impaired ( Peyser et al., 1980 ). Our current results show a similar picture: of the 97 patients referred because of suspected cognitive impairment, 44% (43/97) showed global impairment, a finding that did not differ as a function of disease course. Consequently, our results confirm that standard neuropsychological assessment remains the best way to identify and monitor cognitive impairment in MS patients.
With the substantial increase in research on cognition in MS over the past three decades, and the subsequent increased awareness of cognitive deficits in MS patients, our findings may seem at first glance somewhat counterintuitive; however, there are several factors that likely contributed to the neurologists׳ chance performance. First, MS patients show deficits in metacognition, i.e. their ability to accurately appraise their own cognitive abilities. Diminished insight may be present in up to one-third of MS patients with a progressive disease course, which would result in patients underreporting the presence of cognitive impairment ( Sherman et al., 2008 ). Also, subjective cognitive complaints in MS patients are influenced by other symptoms such as depression and fatigue ( Feinstein et al., 2014 ). Kinsinger et al. obtained subjective ratings of cognitive complaints, depression, fatigue, as well as objective neuropsychological performance in 127 MS patients before and after psychotherapy. Following psychotherapy, there was a reduction in depression and fatigue ratings, which were predictive of changes in subjective cognitive complaints: however, none of these measures were correlated with objective neuropsychological test performance ( Kinsinger et al., 2010 ), suggesting that mood disturbances contribute to a perception of cognitive impairment, which does not necessarily reflect actual cognitive ability. Indeed, Benedict et al. also found that patients׳ ratings of cognitive impairment were significantly correlated with depression ratings on the Beck Depression Inventory, but not with scores on formal neuropsychological testing. Conversely, informant ratings of cognitive impairment were positively correlated with patients׳ cognitive performance ( Benedict et al., 2003 ) (see also Goverover et al, 2005, Kalmar et al, 2008, and O’Brien et al, 2007). If neurologists were therefore referring patients based on subjective complaints of cognitive difficulties, and if only a small minority of patients attend appointments with a family member, this would weaken the overall predictive accuracy. Indeed, in our sample, neurologists׳ accuracy in identifying cognitive impairment was not significantly different from chance. It should be noted that our data did not reflect the potential bias of mood, in that depressed subjects were cognitively more, not less, impaired. However, our sample was restricted to only patients with subjective cognitive complaints, and thus would not include those patients with objective cognitive impairment but no subjective cognitive concerns.
Another possible reason for the clinical examination proving insensitive to cognitive difficulties in MS patients is that cognitive dysfunction in MS is more subtle than that seen in conditions such as Alzheimer׳s or cerebrovascular disease( Al-Khindi et al., 2010 ; McKhann et al., 2011 ; Brand et al., 2014 ). Aphasia, apraxia, and agnosia are not typically encountered in MS patients: rather, the hallmark deficits are those of decreased information processing speed, which may not be readily apparent during a patient encounter. Taken together these factors can explain why widely used cognitive screening instruments such as the Mini-Mental State Examination have such a low sensitivity in detecting impairment in MS ( Beatty and Goodkin, 1990 ).
When interpreting our results relative to those obtained of Peyser et al. 30 years back, it is relevant to note that while the literature pertaining to cognitive impairment in MS has increased exponentially in the interim, and with it the overall awareness amongst neurologists of their patients׳ cognitive challenges, the neurological examination has remained essentially unchanged during this period. Thus, the very examination itself, sensitive to the signs and symptoms of motor, sensory and coordination abnormalities, amongst others, is not geared towards mentation. Seen in this light, it is not too surprising that we have essentially replicated a three decade old finding. What has changed over time, however, is a greater understanding of the frequency, nature and consequences of cognitive deficits in MS patients. The fact that such a high percentage of patients are impaired and as a result struggling with a host of everyday activities adds impetus to the importance of detecting who is impaired. If history taking, the neurological examination and widely used quick screening instruments like the Mini-Mental State Examination lack sensitivity, the emphasis needs to shift elsewhere in overcoming this challenge. The need for brief cognitive assessments with good sensitivity and specificity and which can be completed within the time constraints of a typical clinical visit is now the focus of numerous on-going research endeavors.
For example, the Brief International Cognitive Assessment in Multiple Sclerosis (BICAMS) has been proposed as one solution ( Langdon et al., 2012 ). The BICAMS consists of a test of information processing speed (SDMT), as well as tests for verbal (CVLT-II) and visual memory (BVMT-R), representing the most frequent cognitive deficits observed in MS (Benedict et al, 2006, Glanz et al, 2010, and Van Schependom et al, 2014). An alternative approach advocates the use of brief computerized cognitive assessments. One such screening battery has recently been developed by our group, which takes 10 min to complete and does not rely on motor ability (apart from oral responses), thereby making it suitable for more disabled subjects ( Lapshin et al., 2013 ). It also has good sensitivity across all disease types, including patients with clinical isolated syndromes ( Lapshin et al., 2014 ).
In presenting our findings, we are cognizant of the limitations of the present study. Specifically, we did not obtain neuropsychological assessments in MS patients judged to be cognitively intact, a requirement for calculating the specificity of neurologists׳ clinical impressions. In addition, our study did not systematically control for the types of patients selected for further clinical investigation, which must be considered in terms of the generalizability of our findings. A more stringent design would have randomly selected patients receive both a routine neurological and neuropsychological assessment, with neurologists classifying patients as impaired or intact while blind to neuropsychological test results.
Nonetheless, we present evidence confirming that notwithstanding significant advances in neurologists׳ appreciation of the frequency and burden of cognitive dysfunction in their patients with MS, routine neurological evaluation is insufficient to detect the often subtle cognitive impairments. These data confirm the essential role for some form of neuropsychological inquiry in the assessment of MS patients׳ cognitive function, ideally one that can dovetail with the complementary neurological examination.
Conflict of interest
Kristoffer Romero has no conflicts of interests to disclose.
Prathiba Shammi has no conflicts of interests to disclose.
Anthony Feinstein was supported by a research grant from the MS Society of Canada.
This work was funded by a grant from the MS Society of Canada to Anthony Feinstein.
- Al-Khindi et al., 2010 Al-Khindi T., Macdonald R.L., Schweizer T.A. Cognitive and functional outcome after aneurysmal subarachnoid hemorrhage. Stroke. 2010;41(8):519–536.
- Beatty and Goodkin, 1990 Beatty W.W., Goodkin D.E. Screening for cognitive impairment in multiple sclerosis: an evaluation of the Mini-Mental State Examination. Arch. Neurol. 1990;47:297–301.
- Benedict, 1997 Benedict R.H.B. Brief Visuospatial Memory Test-Revised: Professional Manual. Odessa, Florida: Psychological Assessment Resources; 1997.
- Benedict, 2005 Benedict R.H.B. Integrating Cognitive Function Screening and Assessment into the routine care of multiple sclerosis patients. CNS Spectr. 2005;10(5):384–391.
- Benedict et al., 2006 Benedict R.H.B., Cookfair D., Gavett R., Gunther M., Munschauer F., Garg N., et al. Validity of the Minimal Assessment Of Cognitive Function in Multiple Sclerosis (MACFIMS). J. Int. Neuropsychol. Soc. 2006 12(04):549–558.
- Benedict et al., 2002 Benedict R.H.B., Fischer J.S., Archibald C.J., Peter A., Beatty W.W., Bobholz J., et al. Minimal neuropsychological assessment of MS patients: a consensus approach. Clin. Neuropsychol. 2002;16(3):381–397.
- Benedict et al., 2003 Benedict R.H.B., Munschauer F., Linn R., Miller C., Murphy E., Foley F., et al. Screening for Multiple Sclerosis Cognitive Impairment using a self-administered 15-item questionnaire. 2003;9: 95–101.
- Benedict and Zivadinov, 2011 Benedict R.H.B., Zivadinov, R. Risk factors for and management of cognitive dysfunction in multiple sclerosis. Nat. Rev. Neurol. 2011 7: 332-342.
- Benedict et al., 2005 Benedict R.H.B., Wahlig E., Bakshi R., Fishman I., Munschauer F., Zivadinov R., et al. Predicting quality of life in multiple sclerosis: accounting for physical disability, fatigue, cognition, mood disorder, personality, and behavior change. J. Neurol. Sci. 2005 231(1–2):29–34.
- Benton et al., 1994 Benton A.L., Sivan A.B., Hamsher K., Varney N.R., Spreen O. Contributions of Neuropsychological Assessment. 2nd ed. New York: Oxford University Press; 1994.
- Brand et al., 2014 Brand C., Alber B., Fladung A.-K., Knauer K., König R., Oechsner A., et al. Cognitive performance following spontaneous subarachnoid haemorrhage versus other forms of intracranial haemorrhage. Br. J. Neurosurg. 2014 28(1):68–80.
- Delis et al., 2001 Delis D.C., Kaplan E., Kramer J.H. California Sorting Test. San Antonio: Psychological Corporation; 2001.
- Delis et al., 2000 Delis D.C., Kramer J.H., Kaplan E., Ober B.A. California Verbal Learning Test–second edition. Adult version. Manual. 2000.
- Feinstein et al., 2014 Feinstein A., Magalhaes S., Richard J.-F., Audet B., Moore C. The link between multiple sclerosis and depression. Nat. Rev. Neurol. Nat. Publish. Group; 2014; 10: 510-517.
- Glanz et al., 2010 Glanz B.I., Healy B.C., Rintell D.J., Jaffin S.K., Bakshi R., Weiner H.L. The association between cognitive impairment and quality of life in patients with early multiple sclerosis. J. Neurol. Sci. 2010 290(1–2):75–79.
- Goverover et al., 2005 Goverover Y., Chiaravalloti N., DeLuca J. The relationship between self-awareness of neurobehavioral symptoms, cognitive functions and emotional symptoms in multiple sclerosis.Mult. Scler. J. 2005; 11, 203-212.
- Gronwall, 1977 Gronwall D. Paced auditory serial-addition task: a measure of recovery from concussion. Percept Mot. Skills. 1977;44:367–373.
- Kalmar et al., 2008 Kalmar J.H., Gaudino E.A., Moore N.B., Halper J., DeLuca J. The relationship between cognitive deficits and everyday functional activities in multiple sclerosis. Neuropsychology 2008; 22(4): 442-449
- Kinsinger et al., 2010 Kinsinger S.W., Lattie E., Mohr D.C. Relationship between depression, fatigue, subjective cognitive impairment, and objective neuropsychological functioning in patients with multiple sclerosis. Neuropsychology. 2010;24(5):573–580.
- Langdon, 2010 Langdon D. Multiple sclerosis cognitive impairment in multiple sclerosis – recent advances and future prospects. Eur. Neurol. Rev. 2010;5(1):69–72.
- Langdon et al., 2012 Langdon D.W., Amato M.P., Boringa J., Brochet B., Foley F., Fredrikson S., et al. Recommendations for a brief International Cognitive Assessment For multiple sclerosis (BICAMS). Mult. Scler. J. 201218(6):891–898.
- Lapshin et al., 2014 Lapshin H., Audet B., Feinstein A. Detecting cognitive dysfunction in a busy multiple sclerosis clinical setting: a computer generated approach. Eur. J. Neurol. 2014;21(2):281–286.
- Lapshin et al., 2013 Lapshin H., Lanctôt K.L., Connor P.O., Feinstein A. Assessing the validity of a computer-generated cognitive screening instrument for patients with multiple sclerosis. Mult. Scler. J. 2013;0(0):1–8.
- Mcdonald et al., 2001 Mcdonald W.I., Compston A., Edan G., Goodkin D., Hartung H.Lublin FD, et al , Recommended diagnostic criteria for multiple sclerosis: guidelines from the International Panel on the Diagnosis of multiple sclerosis. Ann. Neurol. 2001;50:121–127.
- McKhann et al., 2011 McKhann G.M., Knopman D.S., Chertkow H., Hyman B.T., Jack C.R., Kawas C.H., et al. The diagnosis of dementia due to Alzheimer׳s disease: recommendations from the National Institute on Aging-Alzheimer׳s Association workgroups on diagnostic guidelines for Alzheimer׳s disease. Alzheimer׳s Dement. Elsevier Ltd.; 2011 7(3):263–269.
- O’Brien et al., 2007 O’Brien A., Gaudino-Goering E., Shawaryn M., Komaroff E., Moore N.B., DeLuca J. Relationship of the multiple sclerosis Neuropsychological Questionnaire (MSNQ) to functional, emotional, and neuropsychological outcomes. Arch. Clin. Neuropsychol. 2007;22(8):933–948.
- Peyser et al., 1980 Peyser J.M., Edwards K.R., Poser C.M., Filskov S.B. Cognitive function in patients with multiple sclerosis. Arch. Neurol. 1980;37:577–579.
- Van Schependom et al., 2014 Van Schependom J., D’hooghe M.B., Cleynhens K., D’hooge M., Haelewyck M.C., De Keyser J., et al. The symbol digit modalities test as sentinel test for cognitive impairment in multiple sclerosis. Eur. J. Neurol. 2014 21(9):1219–1225.
- Sherman et al., 2008 Sherman T.E., Rapport L.J., Ryan K.A. Awareness of deficit in multiple sclerosis. J. Clin. Exp. Neuropsychol. 2008;30(3):301–311.
- Smith, 1982 Smith A. Symbol Digit Modalities Test: Manual. Los Angeles: Western Psychological Services; 1982.
- Zigmond and Snaith, 1983 Zigmond A.S., Snaith R.P. The hospital anxiety and depression scale. Acta Psychiatr. Scand. 1983 67(6):361–370.
a Neuropsychology Consultation Service, Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, Canada M4N 3M5
b Department of Psychiatry, Sunnybrook Health Sciences Centre, 2075 Bayview Avenue, Toronto, Canada M4N 3M5
⁎ Corresponding author at: Sunnybrook Health Sciences Centre, Deptartment of Psychiatry, Room FG52, 2075 Bayview Avenue, Toronto, Canada M4N 3M5. Tel.: +14164804216.
© 2015 Elsevier B.V., All rights reserved.