You are here
A preliminary validation of the brief international cognitive assessment for multiple sclerosis (BICAMS) tool in an Irish population with multiple sclerosis (MS)
Multiple Sclerosis and Related Disorders, Volume 4, Issue 6, November 2015, Pages 521 - 525
Cognitive impairment is common in multiple sclerosis (MS) irrespective of disease stage or subtype. It is typically underreported and neuropsychological testing can be required to detect more subtle evidence of cognitive impairment. The Brief International Cognitive Assessment in Multiple Sclerosis (BICAMS) was an initiative undertaken by a panel of experts with the primary objective of identifying a brief cognitive assessment tool that could be administered by healthcare professionals without formal neuropsychological training to identify early or subtle cognitive impairment among MS patients.
To validate BICAMS in Irish patients with MS and healthy controls.
Consecutive patients attending the MS outpatient department from January to April 2014 were recruited. Age, gender, education, handedness, MS subtype, expanded disability status scale (EDSS) and disease duration were recorded. They were administered BICAMS composed of Symbol Digit Modalities Test (SDMT), California Verbal Learning Test (CVLT-II) and Brief Visuospatial Memory Test (BVMT-R). Depression and anxiety were assessed using the Hospital Anxiety and Depression Scale (HADS). Control participants were composed of unaffected relatives, spouses or carers attending the clinic with a patient and were matched by age, gender and years of education.
Impairment on individual tests was defined as −1.5 SD below reference group means.
67 patients [73% women; mean age: 43.9 yrs (12.1); mean years of education: 13.6 yrs (2.7)] and 66 controls [68% women; mean age 42.7 yrs (12.7); mean years of education: 14.1 yrs (3.2)] were recruited. Of the MS patient group: 70% were classified as having relapsing remitting MS, 28% secondary progressive MS and 2% primary progressive MS (PPMS). Mean EDSS scores were 1.8 (SD: 0.9), 5.7 (SD: 1.4) and 7.0 in each group respectively with mean disease duration of 10.2 (SD: 8.4) years, 20.6 (10.2) and 17 years. Mean scores and standard deviations for patients and control participants respectively were 46 (12.9) and 55.9 (10.9), p<0.001; d=0.83 for SDMT; 45.3 (10.2) and 52.8 (8.8), p<0.001; d=0.79 for CVLT-II and 17.9 (7.1) and 20.7 (6.6), p=0.02; d=0.41 for BVMT-R. Using regression based norms derived from the control sample only 43% of patients compared to 83% of control participants’ results were within the normal range on all three tests.
As expected higher rates of unemployment was seen amongst the patient population compared to control participants. Using the HADS 11 patients were classified as depressed and 13 as suffering from anxiety. Neither, these measures or the level of fatigue as measured by the MFIS was significantly associated with any of the three outcome measures (Pearson r<±0.3).
This study demonstrates that BICAMS is an easy test to administer and should be used as a basic tool to identify patients with cognitive impairment who may benefit from further neuropsychological assessment. Cognitive impairment can put patients at risk of poor self-management of disease including poor mediation adherence, and negatively impact on employment. Once identified appropriate support and monitoring can be put in place. BICAMS may also be used to help guide treatment decisions and rehabilitation. Further studies will be needed to assess its reliability over time and ability to detect meaningful changes.
- Cognitive impairment is common in MS irrespective of disease stage or subtype.
- It affects information processing speed, episodic memory and executive function.
- It can have significant impact on both disease state and psychosocial functioning.
- BICAMS can be used as a screening tool for cognitive impairment in MS.
Keywords: Multiple sclerosis, Cognition, Cognitive impairment, Neuropsychological assessment, Brief International Cognitive Assessment in MS (BICAMS).
Cognitive impairment is common in MS and demonstrable in all disease stages and subtypes. It can have a significant impact on patient's and their care-giver's quality of life. It can influence employment status, physical independence, rehabilitation benefit, treatment adherence and coping resulting in significant societal costs ( Langdon, 2011 ). Cognitive impairment is estimated to affect up to 40% of newly diagnosed people with clinically isolated syndrome (CIS) and relapsing remitting MS (RRMS) ( Amato et al., 2010 ) and up to 60% of those with secondary progressive MS (SPMS) ( Benedict et al., 2006 ). Cognitive decline correlates with imaging parameters in particular evidence of irreversible tissue loss in the form of brain atrophy on MRI seen in both patients with RRMS and SPMS ( Filippi et al., 2010 ).
There is a distinct pattern of cognitive dysfunction associated with MS with information processing speed, episodic memory and executive function most commonly affected ( Langdon, 2011 ). Language is typically spared. Cognitive dysfunction is often underreported by people with MS (PwMS) and may not be recognized by the clinician during their routine consultation. Neuropsychological testing is required to detect subtle cognitive impairment but this is not readily available at all centres and is both costly and time-consuming. Screening tools for other causes of cognitive impairment, such as those used in Alzheimer's disease, correlate poorly with MS cognitive dysfunction (Dagenais et al, 2013 and Swirsky-Sacchetti et al, 1992).
The Brief International Cognitive Assessment in Multiple Sclerosis (BICAMS) was an initiative undertaken by a panel of experts working in the field of MS cognition with the primary objective of identifying a brief cognitive assessment tool that could be administered by healthcare professionals without any formal neuropsychological training to identify early or subtle cognitive impairment ( Langdon et al., 2012 ). Each of the three component parts: Symbol Digit Modalities Test (SDMT) (Smith, 1982 and Smith, 1982), California Verbal Learning Test (CVLT2) ( Delis et al., 2000 ) and Brief Visuospatial Memory Test Revised (BVMT-R) ( Benedict, 1997 ) have previously been shown to have good psychometric properties.
It was considered that a validated measure of cognitive dysfunction could be incorporated into routine clinical practice. This would be used to identify patients at increased risk of poor symptom management, reduced medication adherence and loss of employment allowing them to be appropriately supported and monitored ( Langdon et al., 2012 ). Emphasis was placed on potential for international validation and standardization allowing comparison across different healthcare settings and opening up the possibility of international treatment trials assessing cognitive outcomes. As such, an international validation protocol was devised by the same panel ( Benedict et al., 2012 ) and this program of international validation is currently underway (Dusankova et al, 2012, Eshaghi et al, 2012, and Goretti et al, 2014). With this in mind our study aims to validate this test amongst an English-speaking MS population in the Republic of Ireland.
2. Patients and methods
2.1. Study participants
Patients with a diagnosis of multiple sclerosis ( Polman et al., 2011 ) attending a specialist clinic in a tertiary referral centre between January and April 2014 were invited to participate. The inclusion criteria were: patients over 18 years of age and able to give informed consent. Patients were excluded if they had:
- a. A current or previous neurological disorder other than MS.
- b. Current psychiatric disorder unrelated to their diagnosis.
- c. Co-existent medical condition that might influence cognition.
- d. Previous history of developmental disorder unrelated to MS.
- e. Prior or current history of substance or alcohol dependence/abuse.
- f. Visual, motor or sensory impairment that might interfere with cognitive test performance.
- g. MS relapse and/or corticosteroid treatment within 4 weeks of the assessment.
Control participants were recruited from unaffected relatives, spouses or carers attending the clinic with a patient and were matched by age, gender and years of education.
Full ethical approval for the study was granted by the St. Vincent's University Hospital Ethics Committee.
2.2. Study instruments and procedures
The methodology followed the recommendations for BICAMS national validation ( Benedict et al., 2012 ). In all participants age, sex, handedness, years of education, occupation and employment status was recorded. In the MS group disease subtype, expanded disability status scale (EDSS) ( Kurtzke, 1984 ) and disease duration from symptom onset was also noted. Depression and anxiety was assessed with the Hospital Anxiety and Depression Score (HADS) ( Zigmond and Snaith, 1983 ) and the Modified Fatigue Impact Scale (MFIS) ( Fisk et al., 1994 ) was used to measure fatigue. All participants were administered the BICAMS by a single neurologist (KO'C) and test re-test reliability was confirmed amongst a small subset of patients and controls.
BICAMS is composed of the Symbol Digit Modalities Test (SDMT), California Verbal Learning Test (CVLT-II) and Brief Visuospatial Memory Test Revised (BVMT-R). The SDMT is composed of nine symbols each representing a number 1–9. These pairs are visible in a key at the top of an A4 page. Below there are a number of rows of the same symbols arranged in random order. Participants have to call out the corresponding number and are scored on how many they get right in 90 s. The CVLT-II is composed of a list of 16 words in 4 semantic categories. The examiner reads out the list of words at a steady pace of approximately 20 s. The patient listens to the complete list and is then asked to repeat back as many as possible in any order which the examiner records on a piece of paper. There are 5 trials in total and the score is composed of the total number of words recorded across the 5 trials. On each occasion the patient is asked to remember the answers given on a preceding trial. The BVMT-R consists of 6 abstract symbols on an A4 page. Patients are given 10 s to look at the page, it is then removed from view and they are asked to draw as many symbols as they can remember in the given order on a blank page. These symbols are then scored 0–2 points depending on accuracy and location. This is repeated 3 times and the total score is a summation of all 3 trials.
2.3. Statistical analysis
All statistical analysis was carried out in SPSS version 20. Between group differences were examined with students t-test and χ2 test for continuous and categorical variables respectively. Effect size was calculated with Cohen's d statistic. Results were considered significant at α-level of <0.05. As outlined in previous publications (Parmenter et al, 2010 and Testa et al, 2009), control group scores were converted to scaled scores (M=10, SD=3) using the cumulative frequency of each measure. Regression equations were generated from the control group scaled scores adjusted for age, age2, sex and education and these were used to compute the predicted scores of the MS patients. Their predicted scores were subtracted from their actual scores and divided by the control group's raw residual for each measure generating z-scores. Tests were considered “impaired” with a z-score of ≤−1.5. Test–retest reliability was confirmed amongst a small subset of patients and controls using a Pearson correlation co-efficient with >0.70 considered sufficient.
Sixty-seven patients and 66 control participants were recruited to the study. Baseline characteristics are outlined in Table 1 . Of the MS patient group: 70% were classified as having relapsing remitting MS (RRMS), 28% secondary progressive MS (SPMS) and 2% primary progressive MS (PPMS). Mean EDSS score was 1.8 (SD: 0.9), 5.7 (SD: 1.4) and 7.0 respectively with disease duration of 10.2 (SD: 8.4) years, 20.6 (10.2) and 17 years. As expected higher rates of unemployment was seen amongst the patient population compared to control participants. Using the HADS 11 patients were classified as depressed and 13 as suffering from anxiety. Neither, these measures or the level of fatigue as measured by the MFIS was significantly associated with any of the three outcome measures (Pearson r<±0.3). Analysis of each of the three measures (SDMT, CVLT-II and BVMT-R) as a continuous variable (raw scores) showed significant differences between both patient and control participants ( Table 2 ).
|Patient (n=67)||Control (n=66)|
|Mean (SD)||43.9 (12.1)||42.7 (12.8)||ns|
|Male (%)||18 (27)||21 (32)||ns|
|Female (%)||49 (73)||45 (68)||ns|
|Mean (SD)||13.6 (2.7)||14.1 (3.1)||ns|
|Left (%)||5 (8)||5 (8)||ns|
|Right (%)||62 (92)||61 (92)||ns|
|Not working due to MS/unemployed||29||4||p<0.001|
|Retired/not working by choice||9||7||ns|
SD: standard deviation; ns: not significant.
|Patient (n=67)||Controls (n=66)|
|Mean (SD)||46.0 (12.9)||56.1 (10.6)||p<0.001|
|Effect size r||0.39|
|Mean (SD)||45.3 (10.2)||53.6 (9.1)||p<0.001|
|Effect size r||0.4|
|Mean (SD)||17.9 (7.1)||20.9 (6.5)||p=0.013|
|Effect size r||0.2|
SD: standard deviation; SDMT: Symbol Digit Modalities Test; CVLTII: California Verbal Learning Test second edition; and BVMT-R: Brief Visuospatial Memory Test Revised.
Table 3 outlines the conversion of raw scores to scaled scores (M=10, SD=3) using the cumulative frequency distribution of each measure. Normal control regression models for each of the 3 metrics are outlined in Table 4 . From these models predicted scores for each patient were generated. Their predicted scores were subtracted from the actual scaled score (determined by their performance in each given test) and divided by the SD of the residual of the control scores (SDMT: 2.132; CVLT_II: 2.378; BVMT-R: 2.493) to generate a z-score. A z-score of ≤−1.5 was considered “impaired”.
SDMT: Symbol Digit Modalities Test; CVLTII: California Verbal Learning Test second edition; and BVMT-R: Brief Visuospatial Memory Test Revised.
|Measure||Predictor||B||Standard error B||T||Standardized B||Total R square|
SDMT: Symbol Digit Modalities Test; CVLTII: California Verbal Learning Test second edition; and BVMT-R: Brief Visuospatial Memory Test Revised.
When these variables were categorized as “impaired” and “not impaired” a significant difference was no longer seen between groups on the BVMT-R ( Table 5 ). Of the MS patients 43% scored within the normal range on all 3 tests as opposed to 83% of controls. Impairment on one or more, two or more and three or more tests was seen in 57%, 27% and 5% of MS patients and 17%, 8% and 2% or controls respectively.
|Patient (n=67)||Control (n=66)|
|Not impaired (%)||Impaired * (%)||Not impaired (%)||Impaired * (%)|
⁎ Defined as a z-score of ≤−1.5.
ns: not significant; SDMT: Symbol Digit Modalities Test; CVLTII: California Verbal Learning Test second edition; and BVMT-R: Brief Visuospatial Memory Test Revised
When the MS group was further subdivided into RRMS and progressive MS (SPMS and PPMS) significant between group differences were seen in each of the individual tests. Using the cut off of one or more tests impaired 49% of RRMS and 75% of progressive MS patients met the criterion for cognitive impairment. Similar patterns were seen if the group was subdivided by disease duration with 50% of those diagnosed <10 years (n=30) and 62% diagnosed ≥10 years (n=37) also meeting this criterion. The data set was too small for logistic regression to identify any significant independent predictors and this is likely a combination of age, disability and disease duration.
Test–retest reliability was confirmed at 3 weeks in a small group of patients (n=5) and controls (n=3) with Pearson correlation co-efficient of >0.8 seen in all measures.
This study addressed the recommendations outlined in the international validation protocol for BICAMS ( Benedict et al., 2012 ) and is the first to publish Irish normative data for SDMT, CVLT-II and BVMT-R. Using the criteria of≥1 test abnormal as recommended from previous validations studies ( Dusankova et al., 2012 ; Orchard et al., 2013 ) 57% of our patients fell below the criterion that was indicative of cognitive impairment compared to 17% of age, sex and education matched controls. This suggests the test discriminates well between patients and controls and can be used in clinical practice to identify MS patients who may be experiencing cognitive impairment. The smaller effect size seen in the BVMT-R may represent a cultural phenomenon as its sensitivity and reliability is well established ( Benedict et al., 2006 ). A larger control sample may be needed to establish normative data and these results need to be interpreted with caution.
The strengths of this study were that the test was administered by a neurologist without specialist neuropsychological training and as part of the patient's routine clinic visit as suggested in the protocol ( Benedict et al., 2012 ). The time to administer this test in the department ranged from 10 to 17 min (mean: 12.7). The patient population was a convenience sample and represents a good cross-section of patients with MS. The control population was primarily composed of relatives or carers accompanying the patient to clinic and is therefore likely to be representative of the general population.
The limitations are a relatively small sample size to generate population norms and may explain the differences seen in the BVMT-R. There was no medical history available on the control population and therefore possible confounding effects of medical and other variables cannot be excluded. This group was not screened for depression or anxiety and we were unable to control for this in subsequent analysis; however, no significant correlation was seen between any of the test measures and either the HADS or MFIS score in the MS population and is unlikely to explain the differences seen between the two study groups.
Differences in BICAMS were seen amongst the MS group with respect to MS subtype and disease duration suggesting there may be a linear relationship with time. However this is far from conclusive and serial measurements will be needed to establish reliability and behavior over time and to determine what is a meaningful change.
BICAMS is an easy test to administer which discriminates well between pwMS and controls. The SDMT and CVLT-II are appropriate and valid in the Irish context. Further data is required to establish the BVMT-R's sensitivity. BICAMS should be used as a basic tool to identify patients with cognitive impairment who may benefit from further neuropsychological assessment. Cognitive impairment can put patients at risk of poor self-management of disease including poor mediation adherence, and negatively impact on employment. Once identified appropriate support and monitoring can be put in place. BICAMS may also be used to help guide treatment decisions and rehabilitation. Further studies will be needed to assess its reliability over time and ability to detect meaningful changes.
We would like to thank Bayer HealthCare® for providing us with the assessment tools for this study.
- Amato et al., 2010 M.P. Amato, E. Portaccio, B. Goretti, V. Zipoli, A. Iudice, D. Della Pina, et al. Relevance of cognitive deterioration in early relapsing-remitting MS: a 3-year follow-up study. Mult. Scler.. 2010;16:1474-1482 Crossref
- Benedict, 1997 R.H. Benedict. Brief Visual Menory Test-Revised: Professional Manual. (FPAR, Inc., Odessa, 1997)
- Benedict et al., 2012 R.H. Benedict, M.P. Amato, J. Boringa, B. Brochet, F. Foley, S. Fredrikson, et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC Neurol.. 2012;12:55
- Benedict et al., 2006 R.H. Benedict, D. Cookfair, R. Gavett, M. Gunther, F. Munschauer, N. Garg, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J. Int. Neuropsychol. Soc.: JINS. 2006;12:549-558
- Dagenais et al., 2013 E. Dagenais, I. Rouleau, M. Demers, C. Jobin, E. Roger, L. Chamelian, et al. Value of the MoCA test as a screening instrument in multiple sclerosis. Can. J. Neurol. Sci. J. Can. Sci. Neurol.. 2013;40:410-415 Crossref
- Delis et al., 2000 Delis, D.C.K.D., Kaplan, E., Ober, B.A., 2000. California Verbal Learning Test Manual, Second Ed. In: San Antonio, T.P.C. (Ed.).
- Dusankova et al., 2012 J.B. Dusankova, T. Kalincik, E. Havrdova, R.H. Benedict. Cross cultural validation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) and the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Clin. Neuropsychol.. 2012;26:1186-1200 Crossref
- Eshaghi et al., 2012 A. Eshaghi, S. Riyahi-Alam, T. Roostaei, G. Haeri, A. Aghsaei, M.R. Aidi, et al. Validity and reliability of a Persian translation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS). Clin. Neuropsychol.. 2012;26:975-984 Crossref
- Filippi et al., 2010 M. Filippi, M.A. Rocca, R.H. Benedict, J. DeLuca, J.J. Geurts, S.A. Rombouts, et al. The contribution of MRI in assessing cognitive impairment in multiple sclerosis. Neurology. 2010;75:2121-2128 Crossref
- Fisk et al., 1994 J.D. Fisk, A. Pontefract, P.G. Ritvo, C.J. Archibald, T.J. Murray. The impact of fatigue on patients with multiple sclerosis. Can. J. Neurol. Sci. J. Can. Sci. Neurol.. 1994;21:9-14
- Goretti et al., 2014 B. Goretti, C. Niccolai, B. Hakiki, A. Sturchio, M. Falautano. Eleonora M, et al. The brief international cognitive assessment for multiple sclerosis (BICAMS): normative values with gender, age and education corrections in the Italian population. BMC Neurol.. 2014;14:171 Crossref
- Kurtzke, 1984 J.F. Kurtzke. Disability rating scales in multiple sclerosis. Ann. N. Y. Acad. Sci.. 1984;436:347-360 Crossref
- Langdon, 2011 D.W. Langdon. Cognition in multiple sclerosis. Curr. Opin. Neurol.. 2011;24:244-249 Crossref
- Langdon et al., 2012 D.W. Langdon, M.P. Amato, J. Boringa, B. Brochet, F. Foley, S. Fredrikson, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult. Scler.. 2012;18:891-898 Crossref
- Orchard et al., 2013 A.J. Orchard, G. Giovannoni, D.W. Langdon. UK validation of the Brief International Cognitive Assessment in Multiple Sclerosis (BICAMS). Mult. Scler.. 2013;19:8-597
- Parmenter et al., 2010 B.A. Parmenter, S.M. Testa, D.J. Schretlen, B. Weinstock-Guttman, R.H. Benedict. The utility of regression-based norms in interpreting the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J. Int. Neuropsychol. Soc.: JINS. 2010;16:6-16 Crossref
- Polman et al., 2011 C.H. Polman, S.C. Reingold, B. Banwell, M. Clanet, J.A. Cohen, M. Filippi, et al. Diagnostic criteria for multiple sclerosis: 2010 revisions to the McDonald criteria. Ann. Neurol.. 2011;69:292-302 Crossref
- Smith, 1982 A. Smith. Symbol Digit Modalities Test: Manual. (Western Psychological Services, Los Angeles, 1982)
- Swirsky-Sacchetti et al., 1992 T. Swirsky-Sacchetti, H.L. Field, D.R. Mitchell, J. Seward, F.D. Lublin, R.L. Knobler, et al. The sensitivity of the Mini-Mental State Exam in the white matter dementia of multiple sclerosis. J. Clin. Psychol.. 1992;48:779-786 Crossref
- Testa et al., 2009 S.M. Testa, J.M. Winicki, G.D. Pearlson, B. Gordon, D.J. Schretlen. Accounting for estimated IQ in neuropsychological test performance with regression-based techniques. J. Int. Neuropsychol. Soc.: JINS. 2009;15:1012-1022 Crossref
- Zigmond and Snaith, 1983 A.S. Zigmond, R.P. Snaith. The hospital anxiety and depression scale. Acta Psychiatr. Scand.. 1983;67:361-370 Crossref
a Department of Neurology, St. Vincent's University Hospital, Dublin 4, Ireland
b Department of Psychology, Royal Holloway, University of London, Surrey, UK
c School of Medicine and Medical Sciences, University College Dublin, Ireland
⁎ Corresponding author at: Neurology Department, St Vincent's University Hospital, Elm Park, Dublin 4, Ireand. Fax: +353 12213842.
© 2015 Elsevier B.V., All rights reserved.