You are here
Practice effect in Symbol Digit Modalities Test in multiple sclerosis patients treated with natalizumab
Multiple Sclerosis and Related Disorders, Volume 10, November 2016, Pages 116–122
How practice effect influences cognitive testing measured by monthly Symbol Digit Modalities Test (SDMT) during natalizumab treatment, and what factors confound such effect.
Eighty patients were examined monthly with SDMT for 26.28.4 months. After 26.0±8.1 months, SDMT was also performed with a rearranged key in 59 cases. Results of SDMTs with the rearranged and previous regular key were compared. We examined if gender, age, Extended Disability Status Scale (EDSS), relapses, and disability progression/improvement influence SDMT performed with the regular and the rearranged key, respectively. We also explored if natalizumab applied before regular monthly SDMT may influence practice effect and cognition.
SDMT performance improved by 1.2 points/test during the first six months and by 0.4 points/test thereafter. Rearranging the symbols of the key after 26.0±8.1 months returned SDMT scores to baseline indicating a practice effect. Such practice effect was more significant after longer testing period, but was not influenced by gender, age, relapses, disability progression and prior natalizumab treatment. Although the change from baseline to 2.5 years was significant in subgroups with EDSS 0–3, 3.5–5.5 and 6–7.5, this was higher in patients with EDSS 0–3 compared to 6–7.5.
Practice effect significantly contributes to continuous improvement in SDMT performance during natalizumab treatment: to test cognition, a change in key is required. Practice effect is less pronounced in patients with advanced disease. Cognition remains stable even in patients with progressive disease during more than 2 years of natalizumab treatment indicated by scores corresponding to baseline after changing the key.
- Repeated monthly SDMT score continuously improves in MS patients.
- A new key reverts the SDMT score to baseline indicating a practice effect.
- Change of key is required with repeated use of SDMT in MS patients.
- Practice effect was unrelated to age, relapses and pre-study natalizumab treatment.
- Cognition measured by SDMT is stable on natalizumab treatment over two years.
Keywords: Symbol digit modalities test, Processing speed, Neuropsychological test, Cognition, Changed key, Extended disability status score.
The estimated prevalence of cognitive impairment in MS ranges between 43–70% (Benedict et al, 2006 and Rao et al, 1991) both in the early and late stages of the disease. The effect of cognitive impairment on everyday life activities, employment status, and social relationships is prominent (Amato et al, 1995 and Banati et al, 2010). Cognitive impairment can occur irrespective of disease duration even in the earliest stages of the disease (Achiron and Barak, 2003, Banati et al, 2010, and Glanz et al, 2007) i.e. clinically isolated syndrome (CIS)(Achiron and Barak, 2003; Feuillet et al., 2007; Glanz et al., 2007). Cognitive deficits may develop independently from physical disability and in patients with benign MS (Feuillet et al, 2007, Glanz et al, 2007, and Portaccio et al, 2009). Nevertheless, cognitive disturbances have also been shown to correlate with high EDSS scores, and the presence of cognitive decline may predict a more progressive disease course (Banati et al, 2010, Portaccio et al, 2009, and Zipoli et al, 2010).
Intellectual disability of MS affects various aspects of general cognitive functioning, including efficiency of information processing, verbal and visuo-spatial memory, executive functioning, attention, and visual perceptual processing. all of which are detectable with sensitive neuropsychological test batteries specially developed for the MS population (Benedict et al, 2006 and Rao, 1991). Particularly processing speed and visual memory seem to be most commonly affected (Benedict et al, 2006 and Rao et al, 1991).
The Symbol Digit Modalities Test (SDMT) measures processing speed and working memory. This test is recommended in the Brief International Cognitive Assessment for MS (BICAMS) as the cognitive test of choice, when only 5 min of testing is available. SDMT is easy to administer, it does not require trained personnel (Benedict et al, 2008 and Langdon et al, 2012), and is an effective tool to detect cognitive decline in clinical practice (Van Schependom et al., 2014).
The effect of natalizumab treatment on cognition has been evaluated by SDMT in a few MS cohorts: these indicated improved cognition (Holmen et al, 2011, Iaffaldano et al, 2012, Kunkel et al, 2015, Morrow et al, 2010, and Portaccio et al, 2013). The frequency of SDMT testing and follow-up varies in these studies: monthly, 6-monthly, and annual examinations have been applied for 48 weeks up to 2 years (Holmen et al, 2011, Iaffaldano et al, 2012, Kunkel et al, 2015, Morrow et al, 2010, and Portaccio et al, 2013). Since one of the earliest symptoms of progressive multifocal leukoencephalopathy (PML) associated with natalizumab treatment is subacute cognitive decline, SDMT can also be an efficient and rapid screening test for subclinical PML in patients infected with JC-virus (Sorensen et al., 2012) before monthly natalizumab infusions. However, potential practice effect may complicate interpretation of SDMT results, i.e. patients using the same SDMT every month become familiar with the test and gain practice.
In this study, therefore we examined cognitive performance with monthly SDMT during natalizumab treatment up to 35 months. We evaluated a possible practice effect by a single change in the order of symbols in SDMT after 26.0±8.1 weeks. We also examined the effect of confounding factors, such as age, gender, disability progression, functional status and natalizumab treatment prior regular introduction of SDMT.
2. Materials and methods
2.1. Study population and design
Eighty patients were enrolled, who have been treated with natalizumab at the Department of Neurology, Odense University Hospital, Denmark (Table 1). The study duration ranged from the implementation of SDMT in November 2011 to September 2014.
Study design and demographics.
|Patients treated with natalizumab||80|
|Male gender||23 (29%)|
|Female gender||57 (71%)|
|Patients suffering a relapse during the study||32 (40%)|
|Patients treated with natalizumab before the study||50 (63%)|
|Number of natalizumab infusions before the study||38.416.4|
|Mean number of SDMTs per patient||267a|
|Mean age at first SDMT (years)||41.09.7|
|Mean baseline EDSS||3.32.0|
|Mean EDSS change during follow up||0.12|
a Equals 10428 weeks follow up.
Mean standard deviation is shown where applicable.
Regular SDMT were performed as part of the routine protocol for natalizumab treatment in the region of Southern Denmark. Data were retrospectively reviewed: 2356 individual SDMTs were performed in the study period. The number of consecutive tests per patient varied from 5 to 39, which corresponded to follow up from 4 to 35 months (26.28.4 months). Sixty patients (75%) completed 25 consecutive SDMTs, indicating their participation in the study for at least two years. Demographics are summarized in Table 1.
The study was approved by the Regional Scientific Committees for Southern Denmark and the Danish Data Protection Agency.
2.2. Data collection
SDMT was performed every 4th week before natalizumab infusion. Every test was done in 90 s. Trained nurses collected SDMT data. EDSS scores were determined every six months. For all patients, who had been already treated with natalizumab at the implementation of the SDMT in November 2011, baseline EDSS was the latest performed EDSS, which due to routine EDSS scoring, could at maximum be 6-months old. In September 2014, all clinical and SDMT data were retrieved from the hospital database and from the Danish Multiple Sclerosis Database. Data were stratified according to EDSS, gender, age, relapses and natalizumab treatment before the introduction of SDMT testing. Only the first 30 consecutive SDMTs were included for further analyses. Three cohorts were established based on baseline EDSS: 3, 3.5-5.5 and 6. Three equally sized groups were defined based on the age of the patients. Disability progression was defined as ≥1 point increase of EDSS persisting for at least 6 months retrospectively analyzed at 2 years; improvement was defined as ≥1 point EDSS decrease after 2 years; stable disease was defined if EDSS change was <1 point after 2 years.
2.3. Introduction of a new SDMT key
Nine symbols are paired with numbers in the SDMT test (key). A definite sequence of symbols should be paired with the correct number during a timed (90 s) examination. At the end of the study period in September 2014, all patients still treated with natalizumab (59 patients) were examined with two SDMT, approximately one hour apart with the same nine symbols, but the pairing of symbols and numbers in the key was rearranged. The main sequence of 110 symbols in the test remained the same. At that time point, the number of SDMT testing in this cohort ranged from 8 to 38, with a mean number of SDMT 29.3±8.8.
2.4. Statistical analysis
Statistical analyses were performed using STATA/IC version 13.1 for Windows. The Shapiro-Wilks test for normality was used to ensure legitimate use of parametric statistics. To determine an overall pattern in the improvement of SDMT scores, the mean of every consecutive test was plotted, and analyzed using linear regression.
3.1. Long-term SDMT performance during natalizumab treatment
To evaluate the long-term SDMT performance, we compared the mean score for each consecutive monthly SDMT. This revealed a clear overall improvement, with an increase in mean raw score up to 30 months. This improvement was most rapid during the first six months, indicated by an approximate improvement of 1.2 point per SDMT. Thereafter, a mean improvement of 0.4 point per SDMT was detected (Fig. 1).
Change in SDMT performance during 30 months of natalizumab treatment. SDMT was performed monthly during natalizumab treatment of 80 patients with MS. Mean of monthly SDMT scores and 95% confidence intervals are shown up to 31 consecutive SDMTS, i.e. 30 months.
3.2. Examination of practice effect by the introduction of a new key
The SDMT scores improved significantly even after the 4th consecutive monthly test (p=0.009, n=80) and thereafter (Fig. 1). Since a practice effect due to the frequent SDMT can be responsible for such an early and continuous improvement, we investigated the effect of a rearranged key. In September 2014, 59 patients were examined with a novel key in the SDMT: the symbols were the same as baseline but the order was changed within the key. At that time point, 23 patients were followed for more than 34 months, 22 patients were followed for more than two years, and 14 patients were followed for less than 2 years but for at least 8 months (Fig. 2). Changing the key returned the scores to baseline in all groups. This indicated that a practice effect significantly contributed to the improved SDMT performance. Longer testing with frequent SDMT resulted in a more significant improvement (Fig. 2B). EDSS had no effect on this pattern, except in patients with EDSS6, where changes with the new key were not significant (Fig. 3).
Practice effect on SDMT performance. A. Mean of monthly SDMT scores are shown before introduction of regular monthly SDMT (baseline, white bar), and 2 years after monthly SDMT (black bar). B. A new key was introduced by rearranging symbols. Patients were grouped according to the length of period with monthly SDMT at the time, when the new key was introduced. White bars indicate baseline values, black bars indicate the latest performed SDMT, and grey bars show SDMT scores obtained with the new key. All bars indicate mean.
Effect of EDSS on SDMT scores. A. Patients were stratified into three groups based on EDSS: 0–3.0 (left panel), 3.5–5.5 (middle panel), and 6.0–7.5 (right panel). SDMT scores and mean are shown in the three cohorts: before introduction of regular monthly SDMT (Baseline) to examine effect of functional status on cognitive domains, 2 years after monthly SDMT (Year 2) and after the introduction of a new key with rearranged symbols (New key) to examine the influence on practice effect. B. Mean of monthly SDMT scores and 95% confidence intervals are shown up to 31 consecutive SDMTS, i.e. 30 months within the subgroups of patients with EDSS: 0–3.0, 3.5–5.5 and 6.0–7.5.
We next investigated the effect of several factors on performance with monthly SDMT over 2 years, i.e. their influence on the practice effect and cognition.
3.3. The effect of disease severity, disability progression and improvement
To investigate relationship between disease severity, cognition, and practice effect, we stratified patients according to EDSS and created three cohorts: EDSS 0–3, 3.5–5.5, and 6–7.5. Monthly SDMT performance and effect of changing the key were examined in these cohorts. Significant increase over 2 years in SDMT scores was detected in the groups with the EDSS3 (p<0.001) and EDSS 3.5-5.5 (p=0.001). SDMT scores returned to baseline in these subgroups by using the rearranged SDMT key (Fig. 3). In the group with EDSS 6–7.5, baseline SDMT scores were significantly lower (p<0.001 versus EDSS 0–3, and p=0.005 versus EDSS 3.5–5.5, respectively). In addition, the change from baseline to 2-year SDMT was not significant in this group (Fig. 3B), but became significant after 2.5 years (p=0.03). Nevertheless, the net change of SDMT scores from baseline to 2.5 years was significantly higher in patients with EDSS 0–3 compared to patients with EDSS 6–7.5 (22.68±11.11 and 10.06±5.97, p<0.001). Scores returned to baseline after changing the key even in the cohort of EDSS 6–7.5 (Fig. 3A).
To examine the effect of disability progression or improvement on SDMT performance and learning ability, patients were also stratified according to disability progression (≥1 point increase of EDSS in 2 years, n=19), improvement (≥1 point EDSS decrease in 2 years, n=14) or stable disease (change less than 1 EDSS point, n=46). A significant increase in SDMT scores during 2 years was observed in all three cohorts (p=0.001, p=0.02 and p<0.01 for stable, increased and decreased EDSS, respectively). In addition, SDMT scores returned to baseline in all three cohorts by using the rearranged SDMT key (data not shown).
We also dichotomized the cohort based on a cut-off value below and above 50 of the baseline SDMT, and examined the change of SDMT in the two populations over two years. The mean change of SDMT in patients with baseline SDMT 49 and below was 13.9, while in patients with baseline SDMT 50 and over was 13.4 and this difference was not significant.
3.4. The effect of age, relapse and gender
To investigate if age and disease activity influence cognitive performance and practice effect, we defined three equally sized groups based on age. There was no difference between the youngest and oldest groups comparing SDMT scores at baseline and after 2 years (Fig. 4A). The rate of increase in SDMT was also the same in the two groups (Fig. 4B), and changing the key returned scores to baseline in both groups (Fig. 4A).
Effect of age on SDMT scores. SDMT scores before introduction of regular monthly SDMT (Baseline) and 2 years after monthly SDMT (Year 2) are shown in the youngest and oldest one-thirds of the cohort. A. SDMT scores in the youngest and oldest cohorts at baseline, year 2 and with the new key. B. Slope of increase in SDMT scores during 2 years are shown in the youngest and oldest one-thirds of the cohort.
Thirty-two patients (40%) experienced relapse during the study period. This was not reflected in their SDMT results: no difference at either baseline or after 2 years was observed comparing patients suffering from at least one relapse and patients who did not have relapses (Fig. 5A). Improved SDMT performance was detected in both groups after 2 years, (p<0.01, respectively) (Fig. 5A). Changing the key returned scores to baseline in both groups (Fig. 5A).
Effect of relapses and previous natalizumab treatment on SDMT scores. SDMT scores before introduction of regular monthly SDMT (Baseline), 2 years after monthly SDMT (Year 2) and with the new key are shown. A. SDMT scores in the cohorts of patients who did or did not suffer from relapses. B. SDMT scores in a cohort of patients treated with natalizumab before monthly introduction of SDMT (mean duration of treatment 21.9±20.2 weeks) and in a cohort of patients, who simultaneously started natalizumab and regular SDMT testing.
Gender had no effect on SDMT performance, and changing the key similarly returned SDMT scores to baseline in male and female patients (data not shown).
3.5. The effect of natalizumab treatment prior to regular SDMT testing
We also considered that continuous increase in SDMT may also reflect the effect of natalizumab on cognition or even the practice effect. Therefore, we examined the effect of prior natalizumab treatment on the SDMT performance with the regular and the rearranged key. Patients were stratified into two groups: natalizumab before monthly SDMT, i.e. treatment before November 2011 and natalizumab started together with regular SDMT. After 1 and 1.5 year of SDMT testing, there was a significant increase in SDMT in both groups, respectively (year 1: pretreated p<0.01 n=51, simultaneously treated p=0.03, n=20; year 1.5: pretreated p<0.01, n=50, simultaneously treated p=0.05, n=17)(Figure 6). At 2 years, the change in SDMT remained significant in patients pretreated with natalizumab (p<0.01 n=48), but was not significant in simultaneously treated patients (p=0.09, n=12) (Fig. 5B). Changing the key returned SDMT scores to baseline in both groups (Fig. 5B).
In this study, we examined long-term changes in monthly SDMT performance during natalizumab treatment. We observed a continuous rise in scores up to 30 monthly SDMT over 2.5 years. The increase was most apparent in the first six months of SDMT testing, but mean scores continued to increase by about 0.4 point per test thereafter. Similar continuous increase was also observed in a study with shorter follow up (48 weeks) of monthly testing, and the steepness of the slope also declined over time (Morrow et al., 2010). Testing at every 6 months or annual examination in other studies also resulted in significant improvement after 2 years (Holmen et al, 2011 and Iaffaldano et al, 2012).
We considered that such a change in the rate of increase could be due to two reasons. First, it may be caused by a practice effect, i.e. patients using the same SDMT every month become familiar with the test and gain practice. Second, the possible effect of natalizumab treatment on cognition may reach a plateau and only minor improvements can be seen thereafter. Therefore, (i) we explored the effect of a new key on SDMT performance; (ii) we examined confounding factors related to the disease, and (iii) we addressed the effect of natalizumab administered prior to the introduction of regular, monthly SDMT. A previous paper examined the effect of alternate versions of SDMT with different keys in healthy subjects (Benedict et al., 2012). In contrast, here we performed a longitudinal study in MS patients, and (i) examined the evolution of the learning effect over two years, (ii) verified the learning effect by changing the key after two years, and (iii) related the evolution of the learning effect to demographic and clinical characteristics.
We found that a simple change in the order of the symbols completely reversed the increase in SDMT scores and returned it to baseline values, indicating that practice effect significantly impact SDMT results during natalizumab treatment if frequent testing is applied. Longer testing period resulted in better improvement in SDMT performance, which also supports the contribution of practice effect. Such practice effect may be caused by memorization of test stimuli or familiarity with the process itself. However, a single change after more than 2-year frequent testing should not completely abrogate the effect, if only familiarity plays a role. Patients may also have difficulty to adapt to a new key after repetitive usage of the same SDMT version, and this might influence the outcome with the new key. However, it is unlikely that this would be responsible for such dramatic changes in every case. Indeed by using 5 alternate forms of the SDMT, a previous study did not find noticeable differences (Benedict et al., 2012).
Next, we examined confounding factors, which could influence cognition and the observed practice effect during natalizumab treatment. Disability measured by EDSS influenced SDMT results. First, baseline SDMT scores were significantly lower in patients with EDSS6 compared to cohorts with lower EDSS. This is consistent with earlier findings that higher EDSS scores correlate with worse performance in different neuropsychiatric tests (Amato et al., 1995). Although SDMT scores were significantly higher after 2.5 years in this subgroup, still the net increase in SDMT scores was significantly lower compared to patients with EDSS 0–3, indicating less practice effect. This could be related to continuous cognitive decline per se and/or to a decreased ability to gain practice during SDMT. It is possible that patients with advanced disease are no longer able to increase their processing speed as much as less affected patients. Both grey matter and white matter pathology may contribute to such effects as the disease progresses: cortical lesion number, volume and white matter lesion volume independently predicted the performance of information processing speed and working memory, which is measured by SDMT (Mike et al., 2011; Mike and Illes, 2013). Cortical lesion number also predicted verbal learning and memory (Mike et al., 2011; Mike and Illes, 2013; Roosendaal et al., 2009); grey matter pathology becomes prominent with advanced disease and higher EDSS (Kutzelnigg et al, 2005 and Lassmann, 2007). Cognitive decline predict a more progressive disease course and seem to be more severe in patients with chronic progressive disease compared to those in the relapsing-remitting stage (Patti et al, 2009, Portaccio et al, 2009, and Zipoli et al, 2010). Patients with disability progression, stable disease or improvement also had a significant increase in SDMT scores. But all these cohorts acquired practice effect, indicated by reversal of scores to baseline when using the rearranged key. These data altogether may suggest that patients with higher EDSS may have a deficiency in processing speed, working memory and ability to acquire practice reflected by less increase in SDMT scores after 2.5 years. Nevertheless, there is a practice effect in all EDSS groups.
Four out of ten patients suffered from at least one relapse during the study period, which corresponds to the relapse rate found at two years in the natalizumab phase III trial (Polman et al., 2006). We did not observe a decrease in SDMT scores in patients who experienced relapses at the end of the study period. Previous data suggested 2–3 months temporary decrease (Benedict et al, 2014 and Morrow et al, 2011).
We observed that longer SDMT testing period resulted in improvement of SDMT scores. However, these patients were also treated longer with natalizumab, which might have an effect on the practice effect itself by improving cognition. To examine the effect of natalizumab treatment on the practice effect and cognition, we compared a cohort with treatment before monthly introduction of SDMT (21.920.2 months) to a cohort of patients, who simultaneously started natalizumab and regular SDMT testing. After 1 and 1.5 year, there was a significant increase in SDMT scores in both groups; although the increase was not significant after 2 years in the group simultaneously treated with natalizumab, this can be explained by lack of statistical power due to the decreasing number of patients. Scores reversed to baseline in both cohorts indicating practice effect. Nevertheless, since these scores were not less than baseline values, this indicated that processing speed and working memory did not decline during this period even in patients with EDSS6 or disability progression. Indeed, the effect of natalizumab on cognition has been shown by other cognitive approaches without practice effect (Iaffaldano et al, 2012 and Weinstock-Guttman et al, 2012). Second, the observed continuously improving practice effect during natalizumab treatment may be related to maintained cognitive domains.
In summary, our data indicate that repeated monthly testing with SDMT results in a continuous improvement of SDMT scores up to 35 months during natalizumab treatment, which is less pronounced in patients with more advanced disease. Such improvement in performance is largely attributed to a practice effect. Although SDMT is a convenient screening tool for early signs of PML, if consecutively performed, rearrangement of key could make the results more reliable. Finally, the maintained SDMT scores after 2 years compared to baseline and the continuously improving practice effect up to 38 months could be related to the effect of natalizumab on cognitive domains, but this cannot be formally concluded from this retrospective study due to lack of controls without natalizumab treatment.
This project was supported by grants from Scleroseforeningen (A-19412), Odense University Hospital and Lundbeckfonden (R118-A11472, Denmark).
Conflicts of interest
Dr. Zsolt Illes and Dr. Tobias Sejbaek have received research support from Biogen Idec and travel grants and speaking fees from Bayer Healthcare, Biogen Idec, Merck Serono, Sanofi-aventis/Genzyme, Teva Pharmaceuticals, and Novartis. MB. Malte Roar has no conflicts of interest to declare.
- Achiron and Barak, 2003 A. Achiron, Y. Barak. Cognitive impairment in probable multiple sclerosis. J. Neurol. Neurosurg. Psychiatry. 2003;74(4):443-446 Crossref
- Amato et al., 1995 M.P. Amato, G. Ponziani, G. Pracucci, L. Bracco, G. Siracusa, L. Amaducci. Cognitive impairment in early-onset multiple sclerosis. Pattern, predictors, and impact on everyday life in a 4-year follow-up. Arch. Neurol.. 1995;52(2):168-172 Crossref
- Banati et al., 2010 M. Banati, J. Sandor, A. Mike, E. Illes, L. Bors, A. Feldmann, R. Herold, Z. Illes. Social cognition and Theory of Mind in patients with relapsing-remitting multiple sclerosis. Eur. J. Neurol.. 2010;17(3):426-433 Crossref
- Benedict et al., 2006 R.H. Benedict, D. Cookfair, R. Gavett, M. Gunther, F. Munschauer, N. Garg, B. Weinstock-Guttman. Validity of the minimal assessment of cognitive function in multiple sclerosis (MACFIMS). J. Int Neuropsychol. Soc.. 2006;12(4):549-558
- Benedict et al., 2008 R.H. Benedict, J.A. Duquin, S. Jurgensen, R.A. Rudick, J. Feitcher, F.E. Munschauer, M.A. Panzara, B. Weinstock-Guttman. Repeated assessment of neuropsychological deficits in multiple sclerosis using the Symbol Digit Modalities Test and the MS Neuropsychological Screening Questionnaire. Mult. Scler. (Houndmills Basingstoke Engl.). 2008;14(7):940-946 Crossref
- Benedict et al., 2014 R.H. Benedict, S. Morrow, J. Rodgers, D. Hojnacki, M.A. Bucello, R. Zivadinov, B. Weinstock-Guttman. Characterizing cognitive function during relapse in multiple sclerosis. Mult. Scler. (Houndmills Basingstoke Engl.). 2014;20(13):1745-1752 Crossref
- Benedict et al., 2012 R.H. Benedict, A. Smerbeck, R. Parikh, J. Rodgers, D. Cadavid, D. Erlanger. Reliability and equivalence of alternate forms for the Symbol Digit Modalities Test: implications for multiple sclerosis clinical trials. Mult. Scler. (Houndmills Basingstoke Engl.). 2012;18(9):1320-1325 Crossref
- Feuillet et al., 2007 L. Feuillet, F. Reuter, B. Audoin, I. Malikova, K. Barrau, A.A. Cherif, J. Pelletier. Early cognitive impairment in patients with clinically isolated syndrome suggestive of multiple sclerosis. Mult. Scler. (Houndmills Basingstoke Engl.). 2007;13(1):124-127 Crossref
- Glanz et al., 2007 B.I. Glanz, C.M. Holland, S.A. Gauthier, E.L. Amunwa, Z. Liptak, M.K. Houtchens, R.A. Sperling, S.J. Khoury, C.R. Guttmann, H.L. Weiner. Cognitive dysfunction in patients with clinically isolated syndromes or newly diagnosed multiple sclerosis. Mult. Scler. (Houndmills Basingstoke Engl.). 2007;13(8):1004-1010 Crossref
- Holmen et al., 2011 C. Holmen, F. Piehl, J. Hillert, A. Fogdell-Hahn, M. Lundkvist, E. Karlberg, P. Nilsson, C. Dahle, N. Feltelius, A. Svenningsson, J. Lycke, T. Olsson. A Swedish national post-marketing surveillance study of natalizumab treatment in multiple sclerosis. Mult. Scler. (Houndmills Basingstoke Engl.). 2011;17(6):708-719 Crossref
- Iaffaldano et al., 2012 P. Iaffaldano, R.G. Viterbo, D. Paolicelli, G. Lucchese, E. Portaccio, B. Goretti, V. Direnzo, M. D’Onghia, S. Zoccolella, M.P. Amato, M. Trojano. Impact of natalizumab on cognitive performances and fatigue in relapsing multiple sclerosis: a prospective, open-label, two years observational study. PloS One. 2012;7(4):e35843
- Kunkel et al., 2015 A. Kunkel, M. Fischer, J. Faiss, D. Dahne, W. Kohler, J.H. Faiss. Impact of natalizumab treatment on fatigue, mood, and aspects of cognition in relapsing-remitting multiple sclerosis. Front. Neurol.. 2015;6:97
- Kutzelnigg et al., 2005 A. Kutzelnigg, C.F. Lucchinetti, C. Stadelmann, W. Bruck, H. Rauschka, M. Bergmann, M. Schmidbauer, J.E. Parisi, H. Lassmann. Cortical demyelination and diffuse white matter injury in multiple sclerosis. Brain. 2005;128(Pt 11):2705-2712 Crossref
- Langdon et al., 2012 D.W. Langdon, M.P. Amato, J. Boringa, B. Brochet, F. Foley, S. Fredrikson, P. Hamalainen, H.P. Hartung, L. Krupp, I.K. Penner, A.T. Reder, R.H. Benedict. Recommendations for a Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS). Mult. Scler. (Houndmills Basingstoke Engl.). 2012;18(6):891-898 Crossref
- Lassmann, 2007 H. Lassmann. New concepts on progressive multiple sclerosis. Curr. Neurol. Neurosci. Rep.. 2007;7(3):239-244 Crossref
- Mike and Illes Z, 2013 A. Mike, G.C. Illes Z. Impact of Gray Matter Pathology on Cognitive Function in Multiple Sclerosis. (Neuroimaging - Clinical applications, 2013) (Bright P, pp356-372)
- Mike et al., 2011 A. Mike, B.I. Glanz, P. Hildenbrand, D. Meier, K. Bolden, M. Liguori, E. Dell’Oglio, B.C. Healy, R. Bakshi, C.R. Guttmann. Identification and clinical impact of multiple sclerosis cortical lesions as assessed by routine 3T MR imaging. AJNR Am. J. Neuroradiol.. 2011;32(3):515-521 Crossref
- Morrow et al., 2011 S.A. Morrow, S. Jurgensen, F. Forrestal, F.E. Munchauer, R.H. Benedict. Effects of acute relapses on neuropsychological status in multiple sclerosis patients. J. Neurol.. 2011;258(9):1603-1608 Crossref
- Morrow et al., 2010 S.A. Morrow, P.W. O’Connor, C.H. Polman, A.D. Goodman, L. Kappos, F.D. Lublin, R.A. Rudick, S. Jurgensen, D. Paes, F. Forrestal, R.H. Benedict. Evaluation of the symbol digit modalities test (SDMT) and MS neuropsychological screening questionnaire (MSNQ) in natalizumab-treated MS patients over 48 weeks. Mult. Scler. J.. 2010;16(11):1385-1392 Crossref
- Patti et al., 2009 F. Patti, M.P. Amato, M. Trojano, S. Bastianello, M.R. Tola, B. Goretti, L. Caniatti, E. Di Monte, P. Ferrazza, V. Brescia Morra, S. Lo Fermo, O. Picconi, G. Luccichenti, C.S. Group. Cognitive impairment and its relation with disease measures in mildly disabled patients with relapsing-remitting multiple sclerosis: baseline results from the Cognitive Impairment in Multiple Sclerosis (COGIMUS) study. Mult. Scler. (Houndmills Basingstoke Engl.). 2009;15(7):779-788 Crossref
- Polman et al., 2006 C.H. Polman, P.W. O’Connor, E. Havrdova, M. Hutchinson, L. Kappos, D.H. Miller, J.T. Phillips, F.D. Lublin, G. Giovannoni, A. Wajgt, M. Toal, F. Lynn, M.A. Panzara, A.W. Sandrock, A. Investigators. A randomized, placebo-controlled trial of natalizumab for relapsing multiple sclerosis. N. Engl. J. Med.. 2006;354(9):899-910 Crossref
- Portaccio et al., 2013 E. Portaccio, M.L. Stromillo, B. Goretti, B. Hakiki, A. Giorgio, F. Rossi, A. De Leucio, N. De Stefano, M.P. Amato. Natalizumab may reduce cognitive changes and brain atrophy rate in relapsing-remitting multiple sclerosis--a prospective, non-randomized pilot study. Eur. J. Neurol.. 2013;20(6):986-990 Crossref
- Portaccio et al., 2009 E. Portaccio, M.L. Stromillo, B. Goretti, V. Zipoli, G. Siracusa, M. Battaglini, A. Giorgio, M.L. Bartolozzi, L. Guidi, S. Sorbi, A. Federico, M.P. Amato, N. De Stefano. Neuropsychological and MRI measures predict short-term evolution in benign multiple sclerosis. Neurology. 2009;73(7):498-503 Crossref
- Rao, 1991 Rao, S.M., 1991. A Manual for the Brief Repetable Battery of Neuropsychological Tests in Multiple Sclerosis. New York.
- Rao et al., 1991 S.M. Rao, G.J. Leo, L. Bernardin, F. Unverzagt. Cognitive dysfunction in multiple sclerosis. I. Frequency, patterns, and prediction. Neurology. 1991;41(5):685-691 Crossref
- Roosendaal et al., 2009 S.D. Roosendaal, B. Moraal, P.J. Pouwels, H. Vrenken, J.A. Castelijns, F. Barkhof, J.J. Geurts. Accumulation of cortical lesions in MS: relation with cognitive impairment. Mult. Scler. (Houndmills Basingstoke Engl.). 2009;15(6):708-714 Crossref
- Sorensen et al., 2012 P.S. Sorensen, A. Bertolotto, G. Edan, G. Giovannoni, R. Gold, E. Havrdova, L. Kappos, B.C. Kieseier, X. Montalban, T. Olsson. Risk stratification for progressive multifocal leukoencephalopathy in patients treated with natalizumab. Mult. Scler. (Houndmills Basingstoke Engl.). 2012;18(2):143-152 Crossref
- Van Schependom et al., 2014 J. Van Schependom, M. D’Hooghe, B. Cleynhens, K. D’Hooge, M. Haelewyck, M.C. De Keyser, J. Nagels, G. The Symbol Digit Modalities Test as sentinel test for cognitive impairment in multiple sclerosis. Eur. J. Neurol.. 2014;21(9):1219-1225 (e1271-1212)
- Weinstock-Guttman et al., 2012 B. Weinstock-Guttman, S.L. Galetta, G. Giovannoni, E. Havrdova, M. Hutchinson, L. Kappos, P.W. O’Connor, J.T. Phillips, C. Polman, W.H. Stuart, F. Lynn, C. Hotermans. Additional efficacy endpoints from pivotal natalizumab trials in relapsing-remitting MS. J. Neurol.. 2012;259(5):898-905 Crossref
- Zipoli et al., 2010 V. Zipoli, B. Goretti, B. Hakiki, G. Siracusa, S. Sorbi, E. Portaccio, M.P. Amato. Cognitive impairment predicts conversion to multiple sclerosis in clinically isolated syndromes. Mult. Scler. (Houndmills Basingstoke Engl.). 2010;16(1):62-67 Crossref
© 2016 Elsevier B.V., All rights reserved.