Multiple Sclerosis Resource Centre

Welcome to the Multiple Sclerosis Resource Centre. This website is intended for international healthcare professionals with an interest in Multiple Sclerosis. By clicking the link below you are declaring and confirming that you are a healthcare professional

You are here

Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS): Canadian contribution to the international validation project

Journal of the Neurological Sciences, Volume 362, 15 March 2016, Pages 147–152

Abstract

Background

Given the high prevalence of cognitive dysfunction in people with multiple sclerosis (PWMS) and the lack of availability of specialized neuropsychological services in most MS Clinics, there is a need for a brief cognitive monitoring tool that can be easily administered by MS clinic staff.

Objective

We aimed to establish the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) as a feasible cognitive monitoring tool and provide Canadian data toward the international validation effort. Secondary considerations were to determine if BICAMS correlates with self-reported cognition and predicted vocational status.

Methods

57 PWMS were matched to 51 healthy controls (age, sex, education). Participants completed the BICAMS battery which includes the Symbol Digit Modalities Test, and the learning trials from the California Verbal Learning Test-II and the Brief Visuospatial Memory Test-Revised. Depression, self-reported cognition, and fatigue were assessed. Participants were re-tested 15.6 (SD 2.0) days later.

Results

With impairment defined as “one or more abnormal tests,” 57.9% of MS sample was cognitively impaired. Participants were more likely to be impaired on the BVMT-R (43.9%). On the SDMT and CVLT-II, 28.1% and 26.3% of MS participants were impaired. Sensitivity and specificity were highest for the SDMT. The BICAMS was reliable over time (r value range from 0.69 for BVMT-R to 0.87 for SDMT) with the SDMT being most robust. There was no relationship between BICAMS and subjective cognition. The BVMT-R reliably predicted employment.

Conclusions

The BICAMS detected cognitive impairment to a comparable degree to more comprehensive neuropsychological batteries and is a valid measure of cognition in MS. Reliability of components varies, suggesting care be taken when interpreting serial testing results. The BICAMS is a feasible cognitive assessment tool in Canadians and yields comparable results to other cultures.

Highlights

  • BICAMS is a valid tool for identifying cognitive impairment in MS in Canada.
  • BICAMS is reliable but practice effects should be considered with serial testing.
  • Results in this Canadian study are comparable to results in other countries.

Keywords: Multiple sclerosis, BICAMS, Neuropsychology, Cognition, Validity, Reliability.

1. Introduction

Cognitive dysfunction is present in up to 70% of people with MS [1]. The strong negative impact on health-related quality of life (HRQL) that accompanies cognitive impairment in people with MS (PWMS) [2], [3], [4], [5], and [6] makes it essential to identify the impairment early so that steps can be taken to alleviate some of the associated suffering. Given that specialized neuropsychological services to address this issue are not available in most settings due to resource issues or prohibitive cost, there is a necessity for screening tools that can be used in MS Clinics or physicians' offices to identify those with cognitive impairment. Given the subtleties of the cognitive deficits in some instances, they are not easily identified in standard clinic or physician visits [7]. Thus, the development and implementation of cognitive monitoring tools is gaining widespread interest in the literature given the clear clinical need [8], [9], and [10].

In response to this need, the Brief International Cognitive Assessment for MS (BICAMS) initiative was undertaken [11]. The goal was to bring together a panel of experts in the cognitive aspects of MS to recommend a brief, cognitive assessment for MS that can be used in centers with staff members who may not have neuropsychological training. The tests that were recommended for the BICAMS battery were chosen by the panel on the basis of their psychometric qualities (reliability, validity, and sensitivity), international application, ease of administration, feasibility in the specified context, and acceptability to patients [11]. Particular heed was given to the need for tests that addressed the areas of cognitive deficit that are typically observed in MS. The BICAMS can be administered in 15 min.

Evidence suggests that the primary cognitive deficit in MS may be an impaired ability to process information as quickly as healthy individuals [12], [13], [14], [15], [16], [17], [18], and [19]. Longitudinal studies suggest that information processing speed may be a good predictor of long-term cognitive impairment [20]. Thus, the Symbol Digit Modalities Test (SDMT) [21] was chosen for the BICAMS battery given that it has proven to be a psychometrically sound measure that is sensitive to these deficits and is both easily and rapidly administered. It is more palatable to people with MS [22] than the other commonly used measure of information processing speed in MS (Paced Auditory Serial Addition Test). The SDMT is already included in lengthier batteries designed for MS [23], [24], and [25] and equivalent alternate forms are available [26]. The oral version of the task was selected to ensure that reduced fine motor skills (often associated with MS) does not confound performance.

Memory deficits are also common in MS. These difficulties are primarily during the encoding or learning phase given that retrieval typically reaches normal limits with adequate encoding [27], [28], and [29]. The California Verbal Learning Test-II (CVLT-II) [30] is a standard measure of verbal learning and memory in clinical neuropsychology. Consistent with the expected encoding deficits, the CVLT-II total learning score was found to be sensitive to the memory deficits in MS and was able to discriminate MS patients from healthy controls [25]. Indeed, even when only the first two of the five learning trials were used, verbal memory problems in MS were accurately identified [10]. The third and final BICAMS measure is the Brief Visuospatial Memory Test-Revised (BVMT-R) [31]. This is again a well-established measure of visual learning and memory in MS, has been demonstrated to discriminate well between those with MS and healthy controls [32] and demonstrates good test–retest reliability when used for repeated assessments [33]. As with the CVLT-II, only the learning trials are administered given the good psychometric properties [31] and the demonstrated association with neuroimaging variables measuring lesion volume and atrophy [34], [35], and [36].

International standards for validation have been established [37] and efforts are being made to validate BICAMS across a number of different countries. A Persian translation was developed and validated in Iran [38] on the entire MACFIMS (Minimal Assessment of Cognitive Function in MS) battery (of which BICAMS is a subset of tests) with 158 PWMS with either relapsing–remitting or secondary progressive course matched to 90 healthy controls. All tests were able to discriminate between groups, with ROC analyses demonstrating that the SDMT had the best sensitivity and specificity (area under the curve, AUC = 0.90). The sensitivity of the CVLT-II and the BVMT-R were essentially equivalent (AUC = 0.86 and 0.83, respectively). A similar validation study on a Czech translation [39] was completed with 369 individuals with all subtypes of MS compared to 134 healthy controls. Cognitive impairment was defined as impaired (greater than 1.5 SD from the mean) performance on “one or more of the three BICAMS tests.” This yielded a sensitivity and specificity of 94% and 86%, respectively. Further, the authors investigated the relationship between BICAMS performance and self-reported vocational status. The likelihood of vocational disability increased as the number of BICAMS tests yielding impairment increased. A Hungarian translation also demonstrated group differences between MS and control groups at baseline and follow-up with the exception of the CVLT-II at baseline. Cognitive impairment was documented in 52.3% of their sample using the “one or more abnormal tests” criterion. The Hungarian group documented negative correlations between MS participants' subjective fatigue scores and their cognitive performance on the BICAMS. An Italian version was developed and regression-based norms have been established [40]. The Italian BICAMS identified impairment in 23.9% of the MS sample and yielded a sensitivity of 58.2% and a specificity of 86.7% when impairment was defined with Rao's Brief Repeatable Battery [24]. Finally, an Irish version of the BICAMS [41] yielded impairment on one or more tests in 57% of their sample. The MS and control groups differed on all three BICAMS measures when examining mean raw scores. There was no relationship documented between cognitive impairment and fatigue, depression, or anxiety. Similar studies in other countries are ongoing and results are pending.

In summary, there is a need for a short and easily-administered battery of tests that can identify cognitive impairment in an MS clinic setting. Identification of cognitive impairment will help clinic staff identify those in need of cognitive rehabilitation. The BICAMS battery was developed to be suitable for administration by non-specialized personnel and to possess sound psychometric qualities (i.e. reliability, validity). International collaboration is being fostered so that BICAMS may serve as a viable outcome measure of cognition in clinical trials. International standards for validation have been developed and were applied here [37]. The objectives of the current prospective study were to first, establish BICAMS as a feasible cognitive monitoring tool in the Ottawa MS clinic and second, to provide Canadian data toward the international validation effort to foster global collaboration. We hypothesized that PWMS would perform worse than healthy controls (HC) on BICAMS (criterion-related validity) and that BICAMS would demonstrate high test–retest reliability. Secondary considerations were to determine if BICAMS correlated with self-reported cognition and if BICAMS performance predicts vocational status.

2. Method

2.1. Study population

57 English-speaking individuals with clinically definite MS were matched to 51 English-speaking healthy controls (HC) on age, sex and education. Participants with MS were recruited from the Ottawa Hospital MS Clinic, whereas healthy controls were recruited from the community, family/friends of MS participants (no first degree relatives), and through website advertisements. Participants were between the ages of 18 and 59 and fluent in English. They were excluded if they had any neurological/medical/psychiatric conditions (besides MS and depression) that might impede cognition including prior head trauma or a learning disability. Other exclusion criteria included history of seizures, uncorrected visual acuity problems, corticosteroid or immunosuppressive treatment within 2 months of enrolment, current MS exacerbation and current use of drugs (legal or illegal) which may have an impact on cognitive function.

2.2. Procedures

The study was approved by the Ottawa Health Science Network Research Ethics Board. Participants were compensated for their parking costs, but did not receive any incentives for participation. After undergoing appropriate informed consent procedures and a demographic interview, participants underwent the BICAMS battery, which included the oral SDMT (Rao version) as well as the learning trials from the CVLT-II and the BVMT-R (published versions). Tests were administered in that same fixed order. Depression was measured with the Patient Health Questionnaire-9 (PHQ-9) [42]. This is a freely available tool that is quick to administer and has been validated in MS [43] and [44]. Self-reported cognition was assessed with the Multiple Sclerosis Neuropsychological Questionnaire (MSNQ – informant and self forms), a well validated measure of subjective cognitive deficits that often accompany MS [45]. Fatigue was measured with the Modified Fatigue Impact Scale [46].

Participants were then asked to return for a follow-up session one to three weeks later (mean interval 15.6 (2.0) days; range 9 to 32 days; all numbers between parentheses are standard deviations) to allow for test–retest reliability to be determined. This is the gold standard separation interval when the question is test reliability, controlling for maturation effects. All tests administered in the baseline session were repeated in the same manner and in the same order; however, alternate forms were employed (CVLT-II Alternate form; BVMT-R Form 2; SDMT Form 2) to reduce practice effects from prior exposure to the stimuli.

2.3. Analyses

Before analyses were conducted, the data set was examined and cleaned for possible outlier scores. There was 16.7% of missing values on questionnaire data only (no missing BICAMS data). Data were found to be missing completely at random (Little's test, χ2(1479) = 1357.6, p = .99) and so were imputed using the expectation maximization (EM) method. This process was completed according to accepted and conservative methods for data handling [47].

After evaluating the data for homogeneity of variance, an analysis of co-variance was performed to determine if group differences were statistically significant at the alpha level of .05 while accounting for any differences that might be attributable to depression and fatigue. Test–retest reliability, and the relationship between subjective and objective cognition, were evaluated with Pearson correlational analyses. Paired t-tests were administered to calculate practice effects.

In determining impairment we calculated z scores based on the performance of our control sample. Participants were considered impaired if their score was 1.5 standard deviations below the mean of the control group.

After classifying participants to either a cognitively impaired or cognitively intact group using the criterion of impairment on at least two BICAMS tasks, we employed Receiver-Operating-Characteristic (ROC) analyses to evaluate the classification accuracy of each BICAMS subtest.

Logistic regression [48] was employed to determine if BICAMS performance predicted self-reported vocational status. A stepwise technique was utilized with age, gender, education, depression score, and EDSS score entered as predictors on the first step, and the BICAMS variables entered as predictors on the second step.

3. Results

3.1. Demographic characteristics

Mean age was 45.44 (9.93) for the MS group and 41.92 (10.78) for controls, with no statistically significant difference between groups. Mean years of education was 15.44 (2.68) for the MS group and 16.31 (2.11) for controls, again with no difference between groups. Chi-square analyses demonstrated that the groups were also matched on the proportion of males/females (16/41 for MS; 7/44 for controls). In the MS group the proportion of MS subtypes was reflective of a typical clinic population (44 RRMS (77.2%), 9 SPMS (15.8%), and 4 PPMS (7.0%). Mean EDSS score was 2.70 (1.85) with a range of 0 to 7. Average disease duration was 10.11 (7.72) years.

3.2. Validity

The MS group reported higher levels of depressed affect and fatigue compared to healthy controls (see Table 1). Group differences in the BICAMS variables were examined using ANCOVA (see Table 2). At both baseline and follow-up, the SDMT and the BVMT-R yielded significant group differences with the MS group performing more poorly at both time points. However, no group differences were noted in CVLT-II performance at either time point after accounting for the effects of depression and fatigue.

Table 1 Group differences at baseline and follow-up on measures of depression, fatigue, and self- and informant-reported cognition.

MS HC F Partial Eta Squared p value
PHQ-9 Baseline 5.21 3.02 5.77 0.05 < 0.05
(5.45) (3.78)
Follow-up 5.09 2.76 7.92 0.07 < 0.01
(4.95) (3.38)
mFIS Baseline 29.92 14.96 21.80 0.17 < 0.001
(19.13) (13.26)
Follow-up 30.49 12.95 30.23 0.22 < 0.001
(19.71) (12.07)
MSNQ-S Baseline 17.58 12.67 6.44 0.06 < 0.05
(11.21) (8.57)
Follow-up 16.11 11.04 7.98 0.07 < 0.01
(10.75) (7.36)
MSNQ-I Baseline 19.72 12.87 4.64 0.04 < 0.05
(17.35) (15.48)
Follow-up 15.79 10.19 6.58 0.06 < 0.05
(12.66) (9.60)

PHQ-9 — Patient Health Questionnaire-9; mFIS — Modified Fatigue Impact Scale; MSNQ-S — Multiple Sclerosis Neuropsychological Questionnaire — Self form; MSNQ-I — Multiple Sclerosis Neuropsychological Questionnaire-Informant form. Numbers between parentheses indicate standard deviation.

Table 2 Group differences at baseline and follow-up on BICAMS measures.

MS HC F Partial Eta Squared p value
SDMT Baseline 49.65 59.11 9.80 0.09 < 0.01
(10.78) (8.46)
Follow-up 53.54 65.17 11.52 0.10 < 0.01
(12.50) (9.44)
CVLT-II Baseline 51.63 57.72 3.13 0.03 0.08
(10.05) (7.92)
Follow-up 51.46 57.39 3.22 0.03 0.08
(10.16) (7.67)
BVMT-R Baseline 24.60 29.80 16.03 0.13 < 0.001
(6.54) (3.63)
Follow-up 27.39 30.72 5.59 0.05 < 0.05
(6.26) (3.52)

SDMT — Symbol Digit Modalities Test; CVLT-II — California Verbal Learning Test-II; BVMT-R — Brief Visuospatial Memory Test-Revised. Numbers between parentheses denotes standard deviation. Covariates were depression (PHQ-9) and fatigue (mFIS).

Using the previously reported criteria of impairment defined by “one or more abnormal tests” [39] and [49] we determined that 33 of 57 (57.9%) of our sample of PWMS were cognitively impaired at baseline. Specifically, 28.1%, 19.3% and 10.53% of the MS sample were identified as impaired by one, two, and three tests, respectively at baseline. At follow-up, 28.1%, 8.77%, and 15.8% were impaired by one, two, and three tests, respectively. Our MS participants were more likely to be impaired on the BVMT-R than the other BICAMS subtests with 43.9% meeting criteria for impairment. On the SDMT and CVLT-II, 28.1% and 26.3% of MS participants were impaired, respectively.

3.3. Does BICAMS predict impairment?

We evaluated the classification accuracy of each BICAMS subtest using ROC analyses. Using a criteria for global impairment of − 1.5 SD or below on at least 2 BICAMS tests, the SDMT correctly classified 95.4% of participants and demonstrated the most robust performance for the detection of cognitive dysfunction (Fig. 1). The CVLT-II and BVMT-R correctly classified 81.5% and 81.0% of participants, respectively. Please see Table 3 for suggested cut-off values for each test and their respective sensitivity and specificity.

gr1

Fig. 1 Classification accuracy of BICAMS measures.

Table 3 Sensitivity and specificity of BICAMS tests using recommended cut-offs.

BICAMS test AUC Cut-off Sensitivity Specificity
SDMT 0.954 45.5 97.5% 88.2%
CVLT-II 0.815 47.5 82.5% 70.6%
BVMT-R 0.810 23.5 77.5% 82.4%

SDMT = Symbol Digit Modalities Test; CVLT-II = California Verbal Learning Test-II; BVMT-r = Brief Visuospatial Memory Test-Revised; AUC = Area under the curve.

3.4. Reliability

The test–retest reliability data is presented in Table 4. All three tests demonstrated reliability over time with the SDMT showing the strongest correlation. Strength of correlations was modest to strong (range from 0.69 for BVMT-R to 0.87 for SDMT). As others have demonstrated, modest practice effects were noted on some tests [38]. In particular, the MS group improved on the SDMT and on the BVMT-R between sessions despite the use of alternate forms.

Table 4 Test–retest means, correlations, and practice effect (t-test) calculations for MS group.

Baseline Follow-up Test–retest Paired statistics
Pearson's r p value Mean SD Mean SD t Cohen's d p value
SDMT 0.87 < 0.001 49.65 10.78 53.54 12.50 − 4.83 0.33 < 0.001
CVLT-II 0.74 < 0.001 51.63 10.05 51.46 10.16 0.180 0.01 0.858
BVMT-R 0.68 < 0.001 24.60 6.54 27.39 6.26 − 4.14 0.44 < 0.001

SDMT = Symbol Digit Modalities Test; CVLT-II = California Verbal Learning Test-II; BVMT-R = Brief Visuospatial Memory Test-Revised.

3.5. Relationship between objective and subjective cognition

There was no statistically significant relationship between the BICAMS variables and subjective cognition (as reported by either the participants or their informants) at either baseline or follow-up. Nonetheless, both the self and informant forms of the MSNQ correlated with depression at both baseline (self: r = 0.44, p < 0.001; informant: r = 0.32, p = 0.001) and follow-up (self: r = 0.56, p < 0.001; informant: r = 0.22, p < 0.05). Of note is that both self and informant MSNQ findings did discriminate between the MS and HC groups at both baseline and follow-up (see Table 1).

3.6. BICAMS and vocational status

After accounting for demographic variables and depression in a stepwise logistic regression only two variables were retained in the regression model: One variable from the first block (Expanded Disability Status Scale) and one BICAMS variable from the second block (BVMT-R). The full model significantly predicted employment status (omnibus chi-square = 15.35, df = 2, p < .001). The model accounted for between 23.6% and 32.8% of the variance in employment status, with 86.8% of the employed MS participants successfully predicted. However, only 52.6% of predictions for the unemployed MS participants were accurate. Table 5 gives coefficients and the Wald statistic and associated degrees of freedom and probability values for each of the retained predictor variables. This shows that only BVMT-R reliably predicted employment to a significant degree. The value of the coefficients reveal that an increase of one raw score point on the BVMT-R is associated with a decrease in the likelihood of unemployment by a factor of 0.86.

Table 5 Logistic regression results.

Variables in the equation B Standard Wald df Significance Exp(B) Error
EDSS Scores 0.329 0.184 3.313 1 0.073 1.390
BVMT-R − 0.145 0.054 7.092 1 0.008 0.865
Constant 1.785 1.448 1.519 1 0.218 5.961

4. Discussion

A panel of experts designed the BICAMS as an assessment tool that would identify the typical cognitive deficits in PWMS, allow for monitoring of cognition over time, and could be administered by health care professionals without specific experience in neuropsychological testing. Efforts are being made across the globe to validate the BICAMS across different cultures and languages in order to establish it as a psychometrically sound tool that is internationally applicable. In addition to its clinical utility, it will also be helpful in large-scale international clinical trials as a common outcome measure of cognitive functioning. The current study aimed to evaluate the BICAMS in an English-speaking Canadian sample and establish whether or not it validly discriminated between PWMS and healthy controls, as well as whether or not it demonstrated adequate test–retest reliability. Secondary objectives were to determine if the BICAMS correlated with self-reported cognition and if it predicted vocational status.

The BICAMS tasks were able to collectively identify cognitive impairment in 57.9% of our sample of PWMS using the criteria of impairment defined by one or more abnormal tests. This is comparable to the 58%, 57%, and 52.3% in the Czech, Irish and Hungarian studies, respectively [39], [41], and [49]. It is also comparable to the degree of impairment documented by the more comprehensive MACFIMS battery (55%) [39] and is consistent with rates of cognitive impairment in MS that are consistently reported throughout the literature [1]. The relatively equivalent numbers across cultures certainly supports the international applicability of the BICAMS.

Of the three BICAMS tasks, only the SDMT and the BVMT-R were able to discriminate between PWMS and healthy controls after accounting for depression and fatigue. This was true at both baseline and follow-up. The CVLT-II did not discriminate between groups at either time point. It is notable that this was the measure that demonstrated the least ability to discriminate between groups in the Hungarian study as well [49]. It is possible that the lack of discrimination between groups on the verbal measure may have been related to the fact that our sample was slightly more educated than samples in the other studies (i.e. 15.44 (2.68) years as compared to 14 or less in the other studies). Indeed, education level correlated with CVLT-II score across our entire sample (r = 0.20, p < .05), but did not correlate significantly with either SDMT or BVMT-R performance. In our sample, the PWMS were more likely to be impaired on the BVMT-R than the other BICAMS subtests, with 43.9% meeting criteria for impairment.

The individual BICAMS tasks were each able to accurately classify PWMS into either cognitively impaired or cognitively intact groups with a high degree of accuracy. The method used to define impairment based on results on the same BICAMS tasks is less than ideal, as the preference would be to define impairment on a larger battery of non-BICAMS tasks. Nonetheless, in the absence of such data and using the current methodology, findings were consistent with other studies [25] and [38]. The SDMT demonstrated the highest sensitivity (97.5%) and specificity (88.2%), although both the CVLT-II and BVMT-R were also highly sensitive and specific. The SDMT has consistently been found to be a highly sensitive measure in MS and regularly out-performs the PASAT which is included as the only cognitive outcome measure in the Multiple Sclerosis Functional Composite (MSFC). Our ROC analyses yielded an area under the curve (AUC) of 96.4%. Past studies have yielded similar results with the AUC for the SDMT being 92% and for the PASAT 90% [50], leading to the argument that the SDMT should replace the PASAT in the MSFC. Given the many problems with the PASAT [51] and the fact that patients find it so aversive [22], the inclusion of the more easily administered and palatable SDMT in the BICAMS is more than justified.

When evaluating test–retest reliability of cognitive tests, the goal is to obtain r values of at least 0.70 (considered acceptable) or 0.80 (considered good). In the current study, the only BICAMS measure to meet either of these criteria was the SDMT (r = 0.85). Although both the CVLT-II and the BVMT-R were highly significantly correlated, the values did not reach the level for which one should strive. The same was true for the control sample, with the SDMT being the only measure to attain acceptable reliability. Note that reliability in the current study was calculated over a period of approximately two weeks and that alternate test forms were used for each measure in order to reduce practice effects (thus the test–retest reliability also incorporates alternate-form reliability as well). The Persian study [38] yielded r values between 0.78 and 0.82 for the BICAMS measures. However, an alternate form of the SDMT was not employed at follow-up and the follow-up interval was shorter (i.e. 10.8 (3.78) days) than the current study (15.6 (2.0) days). The Czech and Irish studies [39] and [41] tested participants at only one time point. The Hungarian study [49] assessed test–retest reliability without administering alternate forms at follow-up (i.e. CVLT-II standard form and BVMT-R Form 1 were administered at each visit). In addition, participants received more practice on the SDMT because they completed both written and oral versions. Thus, although they demonstrated moderate to strong test–retest reliability, their decision to re-administer the same forms and to administer both written and oral SDMT contributed to heightened practice effects and likely inflated their r values. Although the finding of higher reliability when using the same forms of each test might then argue for using the same forms at re-test intervals in clinical practice, this would make interpretation of any changes in cognition more challenging as a result of practice. The Italian study also utilized the same forms at re-test (in a healthy control sample only), and like the current study, only the SDMT achieved an r value higher than 0.80 [40].

In the current study where alternate forms were employed, both the SDMT and the BVMT-R demonstrated practice effects with performance being higher at follow-up. This is important to consider when using BICAMS as a cognitive monitoring tool over time. Despite the use of alternate forms, PWMS are able to benefit from simple exposure to the test (i.e. they learn how to learn) regardless of the specific items administered. This may be related to diminished anxiety given familiarity with the testing environment as well as knowing what the parameters are for a particular test. Such effects would be even more pronounced if using the same forms in each test session given that in addition to “learning to learn” the patient would also be familiar with the actual test content as well. Thus, if improvement is noted between sessions during regular clinic monitoring of cognition, one cannot assume that cognition has actually improved. If scores decline over time then we can be more confident that this is indeed true decline. Similarly, if there is no change, this might also be suggestive of decline (given the lack of benefit from practice). In neuropsychological practice, these factors are regularly considered in the context of serial testing. As such, in promoting the BICAMS as a monitoring tool for those less familiar with these psychometric principles, it will be important that initial training on these measures (and their interpretation) includes discussion of these issues.

In this study, although ratings of subjective cognition differed between the MS and HC groups, objective cognitive findings on the BICAMS did not correlate with subjective reports of cognition in either the PWMS themselves, or their informants. The lack of correlation of objective and subjective cognition is a common finding in the literature [52] and [53]. Typically, self forms of the MSNQ correlate more with depression, as was the case here, and informant forms correlate more with objective cognition [54]. The latter was not observed in our sample and suggests that informant ratings were influenced by mood symptoms of the PWMS.

With regard to the ability of the BICAMS to predict vocational status, only the BVMT-R was able to do so after accounting for demographics, depression and disability status. This is in contrast to other work which has demonstrated that the SDMT is more likely to predict vocational outcome [32] or that none of the BICAMS tests alone but rather all tests in the battery together (using the one of three tests criterion) predicted employment status [39]. Despite the differences between these studies, the common element is that some aspect of BICAMS performance is able to predict whether or not a PWMS will be employed or not. As such, this suggests that the BICAMS as a whole has demonstrated ecological validity. Which component of the BICAMS best predicts employment status seems to vary across cultures and may be a line of inquiry worth pursuing in the future.

With regard to the significance of these findings, on a local level, the current project has established the BICAMS as a valid tool for the identification of cognitive impairment in our Ottawa MS population. All tests demonstrated high sensitivity and specificity, with the SDMT demonstrating the most robust findings. In addition, the SDMT demonstrated good test–retest reliability. While the two memory measures were perhaps less valid than would be considered optimal, they did attain statistical significance. The BICAMS was able to identify cognitive impairment at a rate that has already been established as typical even when employing more comprehensive test batteries. The BICAMS also demonstrates ecological validity given that one component (the BVMT-R) was able to predict employment status. As such, findings support the use of the BICAMS in Canadian clinics. Given the less than optimal reliability, clinic staff should receive some training by an expert in psychometric properties of measurement tools to ensure that interpretation of findings is accurate, particularly in the case of cognitive monitoring by serial testing. If the BICAMS is implemented in clinical practice it will allow health care professionals to identify cognitively impaired individuals and in turn, begin to target cognitive dysfunction in interventional studies using cognitive rehabilitation strategies. Research has demonstrated that such strategies positively influence quality of life. This could lead to improvements in clinical practice that would have a direct benefit to patients. Future directions should include the pursuit of a validation study in a Canadian Francophone population. Given the language dialect and cultural differences that exist between European French and Canadian French populations, there is likely a need for more than one French translation.

On a global level, establishing the reliability and validity of the BICAMS in a Canadian sample contributes to the international effort to find a psychometrically sound manner of screening for cognitive impairment in MS that can be used across cultures and languages. A common tool such as this will foster research collaboration worldwide. Having common data points to allow for large-scale studies internationally will help to increase statistical power with larger sample sizes and may prove as a useful clinical outcome measure in future clinical trials.

Acknowledgments

The authors would like to thank the people with MS and the healthy control participants who participated in this study. Their time and effort are much appreciated. The authors would also like to thank the University of Ottawa Brain and Mind Research Institute for generously funding this project. Lastly, thanks goes to Dr. Dawn Langdon for her advice and support during the course of this study.

References

  • [1] M.P. Amato, V. Zipoli, E. Portaccio. Multiple sclerosis-related cognitive changes: a review of cross-sectional and longitudinal studies. J. Neurol. Sci.. 2006;245(1–2):41-46 Crossref
  • [2] J. Benito-Leon, et al. A review about the impact of multiple sclerosis on health-related quality of life. Disabil. Rehabil.. 2003;25:1291-1303 Crossref
  • [3] R. Cutajar, et al. Cognitive function and quality of life in multiple sclerosis patients. J. Neurovirol.. 2000;6(Suppl. 2):S186-S190
  • [4] J.H. Kalmar, et al. The relationship between cognitive deficits and everyday functional activities in multiple sclerosis. Neuropsychology. 2008;22:442-449 Crossref
  • [5] A.J. Mitchell, et al. Quality of life and its assessment in multiple sclerosis: integrating physical and psychological components of wellbeing. Lancet Neurol.. 2005;4(9):556-566 Crossref
  • [6] M.W. Nortvedt, T. Riise. The use of quality of life measures in multiple sclerosis research. Mult. Scler.. 2003;9:63-72 Crossref
  • [7] D.W. Langdon. Cognitive impairment in multiple sclerosis - recent advances and future prospects. Neurol. Rev.. 2010;5:69-72 Crossref
  • [8] R.H.B. Benedict, K. Fuchs. Cognitive dysfunction in multiple sclerosis: New frontiers in assessment and treatment. Int. J. MS Care. 2012;14:55-57
  • [9] F.W. Foley, et al. The need for screening, assessment, and treatment for cognitive dysfunction in multiple sclerosis: Results of a multidisciplinary CMSC consensus conference. Int. J. MS Care. 2010;14:58-64
  • [10] E.S. Gromisch, et al. Using a highly abbreviated California Verbal Learning Test-II to detect verbal memory deficits. Mult. Scler. J.. 2012;:1-4
  • [11] D.W. Langdon, et al. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult. Scler. J.. 2012;18:891-898 Crossref
  • [12] C.J. Archibald, J.D. Fisk. Information processing efficiency in patients with multiple sclerosis. J. Clin. Exp. Neuropsychol.. 2000;22:686-701 Crossref
  • [13] J. DeLuca, et al. Is speed of processing or working memory the primary information processing deficit in multiple sclerosis?. J. Clin. Exp. Neuropsychol.. 2004;26(4):550-562 Crossref
  • [14] H.A. Demaree, et al. Speed of information processing as a key deficit in multiple sclerosis: implications for rehabilitation. J. Neurol. Neurosurg. Psychiatry. 1999;67(5):661-663 Crossref
  • [15] D.R. Denney, et al. Cognitive impairment in relapsing and primary progressive multiple sclerosis: mostly a matter of speed. J. Int. Neuropsychol. Soc.. 2004;10(7):948-956
  • [16] C. Forn, et al. Information-processing speed is the primary deficit underlying the poor performance of multiple sclerosis patients in the Paced Auditory Serial Addition Test (PASAT). J. Clin. Exp. Neuropsychol.. 2008;30(7):789-796 Crossref
  • [17] R. Kail. Speed of information processing in patients with multiple sclerosis. J. Clin. Exp. Neuropsychol.. 1998;20(1):98-106 Crossref
  • [18] J. Lengenfelder, et al. Processing speed interacts with working memory efficiency in multiple sclerosis. Arch. Clin. Neuropsychol.. 2006;21:229-238 Crossref
  • [19] S.M. Rao, P. St Aubin-Faubert, G.J. Leo. Information processing speed in patients with multiple sclerosis. J. Clin. Exp. Neuropsychol.. 1989;11(4):471-477 Crossref
  • [20] G. Bergendal, S. Fredrikson, O. Almkvist. Selective decline in infomation processing in subgroups of multiple sclerosis. Eur. Neurol.. 2007;57:193-202 Crossref
  • [21] A. Smith. Symbol Digit Modalities Test. (Western Psychological Services, Los Angeles, 1982)
  • [22] L.A. Walker, et al. Tests of information processing speed: what do people with multiple sclerosis think about them?. Int. J. MS Care. 2012;14(2):92-99 Crossref
  • [23] R.H.B. Benedict, et al. Minimal neuropsychological assessment of MS patients: a concensus approach. Clin. Neuropsychol.. 2002;16:381-397 Crossref
  • [24] S.M. Rao. A Manual for the Brief Repeatable Battery of Neuropsychological Tests in Multiple Sclerosis. (National MS Society, New York, 1991)
  • [25] L. Strober, et al. Sensitivity of conventional memory tests in multiple sclerosis: comparing the Rao Brief Repeatable Neuropsychological Battery and the Minimal Assessment of Cognitive Function in MS. Mult. Scler. J.. 2009;15:1077-1084 Crossref
  • [26] R.H.B. Benedict, et al. Reliability and equivalence of alternate forms for the Symbol Digit Modalities Test: implications for multiple sclerosis clinical trials. Mult. Scler. J.. 2012;18:1320-1325 Crossref
  • [27] J. DeLuca, S. Barbieri-Berger, S.K. Johnson. The nature of memory impairments in multiple sclerosis: acquisition versus retrieval. J. Clin. Exp. Neuropsychol.. 1994;16(2):183-189 Crossref
  • [28] J. DeLuca, et al. Acquisition and storage deficits in multiple sclerosis. J. Clin. Exp. Neuropsychol.. 1998;20:376-390
  • [29] E.A. Gaudino, et al. A comparison of memory performance in relapsing–remitting, primary progressive and secondary progressive, multiple sclerosis. Neuropsychiatry Neuropsychol. Behav. Neurol.. 2001;14(1):32-44
  • [30] D.C. Delis, et al. California Verbal Learning Test - Second Edition. (Psychological Corporation, San Antonio, Texas, 2000)
  • [31] R.H.B. Benedict. The Brief Visual Memory Test-Revised. (Psychological Assessment Resources, Lutz, Florida, 1997)
  • [32] R.H.B. Benedict, et al. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J. Int. Neuropsychol. Soc.. 2006;12:549-558
  • [33] R.H.B. Benedict. Effects of using same- versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis. J. Int. Neuropsychol. Soc.. 2005;11:727-736
  • [34] R.H.B. Benedict, et al. Frontal cortex atrophy predicts cognitive impairment in multiple sclerosis. J. Neuropsychol. Clin. Neurosci.. 2002;14:44-51
  • [35] M.K. Houtchens, et al. Thalamic atrophy and cognition in multiple sclerosis. Neurology. 2007;18:1213-1223 Crossref
  • [36] R.H.B. Benedict, et al. Memory impairment in multiple sclerosis: correlation with deep grey matter and mesial temporal atrophy. J. Neurol. Neurosurg. Psychiatry. 2009;80:201-206 Crossref
  • [37] R.H.B. Benedict, et al. Brief International Cognitive Assessment for MS (BICAMS): international standards for validation. BMC Neurol.. 2012;12:1-8
  • [38] A. Eshaghi, et al. Validity and reliability of a Persian translation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS). Clin. Neuropsychol.. 2012;26(6):975-984 Crossref
  • [39] J.B. Dusankova, et al. Cross cultural validation of the Minimal Assessment of Cognitive Function in Multiple Sclerosis (MACFIMS) and the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Clin. Neuropsychol.. 2012;26(7):1186-1200 Crossref
  • [40] B. Goretti, et al. The Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS): normative values with gender, age and education corrections in the Italian population. BMC Neurol.. 2014;14(171):1-6
  • [41] K. O'Connell, et al. A preliminary validation o the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) tool in an Irish population with multiple sclerosis (MS). Multiple Sclerosis and Related Disorders (, 2015)
  • [42] B. Lowe, et al. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med. Care. 2004;42(12):1194-1201
  • [43] S.B. Patten, et al. Validity of four screening scales for major depression in MS. Mult. Scler. J.. 2015;:1-8
  • [44] D. Amtmann, et al. Comparing CESD-10, PHQ-9, and PROMIS Depression instruments in individuals with multiple sclerosis. Rehabil. Psychol.. 2014;59(2):220-229 Crossref
  • [45] R.H.B. Benedict, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Mult. Scler. J.. 2003;9:95-101 Crossref
  • [46] J.D. Fisk, et al. Measuring the functional impact of fatigue: Initial validation of the Fatigue Impact Scale. Clin. Infect. Dis.. 1994;18(Suppl. 1):S79-S83 Crossref
  • [47] B.G. Tabachnick, L.S. Fidell. Using Multivariate Statistics. (Pearson Education Inc., Boston, MA, 2007)
  • [48] J. Desjardins. L'analyse de regression logistique. Tutor. Quant. Methods Psychol.. 2005;1(1):35-41
  • [49] D. Sandi, et al. The Hungarian validation of the Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) battery and the correlation of cognitive impairment with fatigue and quality of life. Mult. Scler. Relat. Disord.. 2015;
  • [50] A.S. Drake, et al. Psychometrics and normative data for the Multiple Sclerosis Functional Composite: replacing the PASAT with the Symbol Digit Modalities Test. Mult. Scler. J.. 2010;16(2):228-237 Crossref
  • [51] T.N. Tombaugh. A comprehensive review of the Paced Auditory Serial Addition Test. Arch. Clin. Neuropsychol.. 2006;21:53-76 Crossref
  • [52] L.S. Middleton, et al. The relationship between perceived and objective cognitive functioning in multiple sclerosis. Arch. Clin. Neuropsychol.. 2006;21(5):487-494 Crossref
  • [53] Y. Goverover, N. Chiaravalloti, J. DeLuca. The relationship between self-awareness of neurobehavioral symptoms, cognitive functioning, and emotional symptoms in multiple sclerosis. Mult. Scler.. 2005;11(2):203-212 Crossref
  • [54] A. O'Brien, et al. Relationship of the Multiple Sclerosis Neuropsychological Questionnaire (MSNQ) to functional, emotional, and neuropsychological outcomes. Arch. Clin. Neuropsychol.. 2007;22:933-948 Crossref

Footnotes

a University of Ottawa Brain and Mind Research Institute, Canada

b The Ottawa Hospital Research Institute, Canada

c University of Ottawa, Faculty of Medicine, Canada

d University of Ottawa, School of Psychology, Canada

e Carleton University, Department of Psychology, Canada

f Carleton University, Institute of Cognitive Science, Canada

g St. Paul University, Canada

Corresponding author at: The Ottawa Hospital, 501 Smyth Road, Suite 7300, Ottawa, Ontario K1H 8L6, Canada.


Search this site

Stay up-to-date with our monthly e-alert

If you want to regularly receive information on what is happening in MS research sign up to our e-alert.

Subscribe »

About the Editors

  • Prof Timothy Vartanian

    Timothy Vartanian, Professor at the Brain and Mind Research Institute and the Department of Neurology, Weill Cornell Medical College, Cornell...
  • Dr Claire S. Riley

    Claire S. Riley, MD is an assistant attending neurologist and assistant professor of neurology in the Neurological Institute, Columbia University,...
  • Dr Rebecca Farber

    Rebecca Farber, MD is an attending neurologist and assistant professor of neurology at the Neurological Institute, Columbia University, in New...

This online Resource Centre has been made possible by a donation from EMD Serono, Inc., a business of Merck KGaA, Darmstadt, Germany.

Note that EMD Serono, Inc., has no editorial control or influence over the content of this Resource Centre. The Resource Centre and all content therein are subject to an independent editorial review.

The Grant for Multiple Sclerosis Innovation
supports promising translational research projects by academic researchers to improve understanding of multiple sclerosis (MS) for the ultimate benefit of patients.  For full information and application details, please click here

Journal Editor's choice

Recommended by Prof. Brenda Banwell

Causes of death among persons with multiple sclerosis

Gary R. Cutter, Jeffrey Zimmerman, Amber R. Salter, et al.

Multiple Sclerosis and Related Disorders, September 2015, Vol 4 Issue 5