You are here
Predictors and impact of the working alliance in the neuropsychological rehabilitation of patients with multiple sclerosis
Journal of the Neurological Sciences
There is preliminary evidence of positive effects of neuropsychological rehabilitation in multiple sclerosis (MS). However, whether a working alliance affects rehabilitation outcome has not been studied.
The aim of this study was to evaluate the effects of the baseline patient-related (cognitive, mood and fatigue symptoms, cognitive status, demographic factors) and illness-related factors (duration and severity of the disease) on the alliance, as well as the effects of the alliance on rehabilitation outcome in neuropsychological rehabilitation among MS patients.
Fifty-six patients with relapsing–remitting MS received multimodal neuropsychological intervention (attention retraining, learning strategies, psychoeducation, psychological support, homework assignments) conducted once a week in 60-minute sessions for thirteen consecutive weeks. After the intervention, both patients and therapists evaluated the alliance with the short form of the Working Alliance Inventory.
None of the baseline factors was related to the alliance. Better patient-evaluated alliance was associated with a more prominent decrease in fatigue symptoms and greater achievement of rehabilitation goals. Better therapist-evaluated alliance was associated with greater benefit from the intervention as evaluated by therapists.
A positive patient–therapist alliance may relate to positive neuropsychological rehabilitation outcome in MS.
Keywords: Alliance, Multiple sclerosis, MS, Neuropsychological rehabilitation, Outcome, Working alliance.
The concept of a working alliance (referred to later in the text as the alliance) is rooted in psychotherapy and has been studied extensively in that field, showing that alliance in psychotherapy is a critical therapeutic element for a desirable outcome  . The most commonly used definition for alliance is that of Bordin  : The working alliance is the combination of 1) client and therapist agreement on goals; 2) client and therapist agreement on the utility and efficacy of the things done in rehabilitation; and 3) the development of a personal bond between the participants. Horvath and Greenberg  developed the Working Alliance Inventory (WAI) with the purpose of measuring Bordin's three factors. Since then it has become the most widely used measure of alliance and as such its reliability and validity have been repeatedly established and .
It has been shown that a positive alliance is associated with positive outcomes in psychotherapeutic interventions for variables such as mood, anxiety, interpersonal problems, and general psychological functioning. Over the past few decades, meta-analytic overviews have reported moderate but robust alliance–outcome correlations and . A good working alliance has also been found to relate to positive outcome in the chronic care of psychiatric patients  and in physical rehabilitation  .
Cognitive dysfunction is a common manifestation among patients with multiple sclerosis (MS) occurring in about 50–60% of patients and having widespread effects on quality of life and . Cognitive functions most affected are speed of information-processing, memory, executive skills and complex attention and . Depression and fatigue are also common in MS and may aggravate cognitive symptoms and . According to systematic reviews, there is preliminary evidence of positive effects of neuropsychological rehabilitation in MS and . However, the factors contributing to rehabilitation outcome are not well known and additional research is called for to investigate the factors that influence the effectiveness of the rehabilitation  .
The role of the alliance in neuropsychological rehabilitation has not been widely studied. Previous findings on traumatic brain injury patients show a positive relationship between alliance and employment/productivity status and , metacognitive skills  , reduction of depressive symptoms  , patient's experience of success  , and driving clearance and ability  . In MS, a positive alliance has been found to predict reduction in depressive symptoms after cognitive–behavioural-therapy  but, to our knowledge, the significance of an alliance in neuropsychological rehabilitation in MS has not been studied. The aims of our study were twofold. Firstly, we evaluated whether patient-related baseline factors (cognitive, mood or fatigue symptoms, cognitive status or demographic factors) or illness-related factors (duration or severity of the disease) have an effect on the alliance in neuropsychological rehabilitation in MS. Secondly, we evaluated the effects of the patient–therapist alliance on rehabilitation outcome.
2. Materials and methods
Complete details of the study procedure, the patients included, the intervention, and the outcome measures employed are described in our previous publication  . Briefly, a total of 102 patients with clinically definite  relapsing–remitting MS were included in the study. The inclusion criteria were clinically definite relapsing–remitting MS, the Expanded Disability Status Scale (EDSS  < 6), subjective deficits in attention (total score of questions 1, 2, and 11 in Multiple Sclerosis Neuropsychological Questionnaire-Patient, MSNQ-P  ≥ 6), objective deficits in information processing speed (Symbol Digit Modalities Test, SDMT  total score ≤ 50), and ages 18–59. Patients with a history of drug or alcohol abuse, psychiatric disorder, acute relapses, neurological disease other than MS, or ongoing neuropsychological rehabilitation were excluded. All patients provided written informed consent, and the study protocol was approved by the Ethics Committee of Seinäjoki Central Hospital, Tampere University Hospital and Turku University Hospital.
Patients were randomised into an intervention and a control group using a computer-generated random number table and stratified randomisation according to age (18–37 vs. 38–58 years), gender (female vs. male), years of education (< 12 vs. ≥ 12 years), disability (EDSS 0–4 vs. 4.5–5.5), and study centre, in the ratio 3:2 (intervention:control) by an independent statistician. Of the 102 randomised patients, 98 (intervention group 58, control group 40) completed the whole study. In the present study, we were interested in the working alliance between therapist and patient and, therefore, the study sample was solely the intervention group. Because of two drop-outs not fulfilling the alliance evaluations, the number of included patients was 56 ( Table 1 ).
|Age in years||43.7||8.8||22–58|
|Education in years||13.5||2.3||8–18|
|Duration since MS diagnosis in years||9.3||6.7||1–32|
|Verbal memory (Z-score)||− 1.7||2.1||− 6.7 to 1.4|
|Visual memory (Z-score)||− 0.02||1.0||− 2.3 to 1.5|
|Attention-executive functions (Z-score)||− 1.1||1.0||− 3.6 to 0.8|
|Fluency (Z-score)||0.6||2.0||− 2.7 to 5.9|
|PDQ, total score||36.3||12.0||9–65|
|BDI-II, total score||12.9||7.2||0–29|
|MSIS-psychological, subscale score||31.5||20.0||0–81|
|FSMC, total score||64.7||18.0||25–98|
|FSMC-cognitive, subscale score||32.7||9.0||12–49|
EDSS = Expanded Disability Status Scale; PDQ = Perceived Deficits Questionnaire; BDI-II = Beck Depression Inventory II; MSIS-psychological = Multiple Sclerosis Impact Scale—psychological composite; FSMC = Fatigue Scale for Motor and Cognitive Fatigue.
All patients in the intervention group received outpatient neuropsychological rehabilitation conducted once a week in 60-minute sessions for thirteen consecutive weeks. Patients were instructed to avoid other neuropsychological interventions during the study. None of the patients received neuropsychological rehabilitation, some patients received out-patient physiotherapy or an in-patient rehabilitation period. 95.5% of the intervention appointments were realised as planned. The rehabilitation consisted of a computer-based attention and working memory retraining, learning compensatory strategies, psychoeducation, and homework assignments connected with rehabilitation goals, as well as psychological support to promote coping with cognitive impairments. The rehabilitation neuropsychologists (n = 3) were not the same as the assessing neuropsychologists (n = 3). The assessing neuropsychologists were blind to group membership. After the last neuropsychological assessment, the assessing neuropsychologists' estimate of patients' group membership (intervention or control) was correct in 62% of the cases. At the beginning of the intervention, patients set goals for the rehabilitation together with the neuropsychologist using the Goal Attainment Scaling (GAS)  . Every patient was asked to set one to three goals concerning attentional problems they faced in everyday life.
2.3. Outcome measures
Fig. 1 illustrates the research design. The blinded neuropsychologists performed neuropsychological assessments at baseline, after three months (end of intervention) and after six months. The cognitive performance was evaluated with the Brief Repeatable Battery of Neuropsychological Tests (BRBNT), a widely used brief neuropsychological battery with reasonable availability and acceptable sensitivity in MS and . The BRBNT includes: the Buschke Selective Reminding Test (BSRT) to assess verbal memory; the 10/36 Spatial Recall Test (10/36) to assess visual memory; the Symbol Digit Modalities Test (SDMT) to assess information processing speed and executive functions; the Paced Auditory Serial Addition Test 2 and 3 seconds (PASAT 2–3) to assess attention, information processing speed, and working memory; and the Controlled Oral Word Association Test (COWAT) to assess semantic fluency.
The self-perceived cognitive deficits were evaluated with the Perceived Deficits Questionnaire (PDQ)  , the self-perceived depressive symptoms with the Beck Depression Inventory II (BDI-II)  , the psychological impact of the disease with the Multiple Sclerosis Impact Scale (MSIS-29)  , the self-perceived feelings of fatigue with the Fatigue Scale for Motor and Cognitive Fatigue (FSMC)  , and achievement of personal rehabilitation goals with the Goal Attainment Scaling (GAS)  . Previous studies provide good evidence for the reliability and validity of the methods used, like the PDQ  , the MSIS-29  , the FSMC  , and the GAS  .
At the end of the intervention, before the last neuropsychological assessment, the working alliance was evaluated separately and independently by the patient and the therapist with the short form of the Working Alliance Inventory (WAI) and . In addition to the general alliance factor (total score), the task factor (patient–therapist agreement on the utility and efficacy of the things done in rehabilitation; questions 2, 4, 24, 35), the bond factor (personal bond between patient and therapist; questions 8, 21, 23, 26), and the goal factor (patient–therapist agreement on goals; questions 12, 22, 27, 32) were analysed according to Tracey and Kokotovic  . The maximum total score in WAI was 84. Furthermore, after the intervention, before knowing the results of the neuropsychological assessment, the therapists evaluated the benefit patients had received from the intervention using a four-point scale: 1) not at all, 2) to some extent, 3) moderately, and 4) obviously.
2.4. Data analyses
Paired samplest-tests and Wilcoxon Signed Rank Tests were employed to measure differences between the alliance ratings of therapist and patient. Statistical comparisons between the severity of the disease and the alliance ratings were computed using Mann–WhitneyU-tests and Student'st-tests. When the therapist's estimate of benefit (moderate, some extent, obvious) was compared to the alliance ratings, the analysis of variance (ANOVA) and the Kruskal–Wallis test were used. The Tukey honest significance difference test was used for post hoc pairwise comparisons following ANOVAs, and Mann–Whitney comparisons following Kruskal–Wallis tests.
In order to obtain composite scores to different cognitive domains, Z-scores for each cognitive domain (Z verbal memory, Z visual memory, Z attention-executive functions, and Z fluency; see equations below) were created  . To obtain Z-scores, a reference group of 24 healthy controls was used  .
The rehabilitation outcomes were the mean change between the follow-up (after six months) and the baseline assessment in cognitive domains and self-report questionnaires, as well as the T-score on the GAS (see Fig. 1 ). Parametric (Pearson's) and non-parametric (Spearman's) correlation was used to evaluate the association between alliance and baseline factors, as well as between alliance and rehabilitation outcome.
3.1. WAI ratings between patients and therapists
The overall alliance ratings differed significantly between the patients and the therapists: the patients evaluated the alliance as better than the therapists (mean WAI total score 77.2 SD (4.6), range 67–84 vs. mean WAI total score 70.7 SD (6.3), range 53–81,p < 0.001). Similarly, the patients reported stronger agreement on the utility and efficacy of the things done in rehabilitation (mean WAI task 25.5 (1.9) vs. 23.1 (2.4),p < 0.001), stronger personal bond (mean WAI bond 25.3 (2.0) vs. 23.9 (2.2),p < 0.001), and stronger agreement on goals (mean WAI goal 26.4 (1.6) vs. 23.7 (2.0),p < 0.001) than the therapists. However, the correlation of the overall alliance ratings between the patients and the therapists was significant (WAI total scorer = 0.418,p = 0.001), as were the correlations between the patients and the therapists in agreement on the utility and efficacy of the things done in rehabilitation (WAI taskr = 0.373,p = 0.005), in personal bond (WAI bondr = 0.302,p = 0.024), and in agreement on goals (WAI goalr = 0.355,p = 0.007).
3.2. Relationship between working alliance and baseline variables
The correlations between alliance ratings and baseline factors are summarised in Table 2 . Neither the cognitive status of the patients nor self-reported cognitive, depressive or fatigue symptoms at baseline correlated significantly with the therapist-evaluated alliance. However, the perceived psychological burden correlated with the therapist-evaluated alliance: the therapists evaluated the alliance as better in those patients who reported less burden at baseline.
|WAI therapist||WAI patient|
|Task a||Bond a||Goal a||Total||Task||Bond||Goal a||Total|
|Verbal memory||r||− 0.029||− 0.206||− 0.079||− 0.033||− 0.105||− 0.161||− 0.182||− 0.178|
|Visual memory||r||− 0.096||− 0.187||− 0.075||− 0.087||− 0.251||− 0.144||− 0.264||− 0.251|
|Attention-executive functions||r||0.088||− 0.044||0.213||0.125||− 0.185||− 0.278||− 0.132||− 0.251|
|Fluency||r||− 0.037||− 0.056||0.041||0.001||− 0.175||0.039||− 0.179||− 0.104|
|PDQ, total score||r||0.095||0.179||0.198||0.180||0.020||0.099||0.133||0.107|
|BDI-II, total score||r||0.099||0.196||0.231||0.179||0.018||0.101||0.267||0.128|
|MSIS-psyc, composite score||r||0.216||0.272||0.325||0.276||0.087||0.172||0.205||0.159|
|FSMC, total score||r||− 0.003||0.114||0.132||0.091||− 0.105||− 0.080||0.041||− 0.057|
|FSMC-cognitive, composite score||r||− 0.099||0.066||0.053||0.041||− 0.119||− 0.093||0.064||− 0.059|
|Education years||r||0.207||0.086||0.183||0.151||− 0.072||0.120||0.011||0.031|
a Spearman's correlation coefficient; in other variables Pearson's correlation coefficient.
Bold values indicate significance at p < 0.05. Scores were transformed so that a larger value always indicates better performance.
PDQ = Perceived Deficits Questionnaire; BDI-II = Beck Depression Inventory II; MSIS-psyc = Multiple Sclerosis Impact Scale—psychological dimension; FSMC = Fatigue Scale for Motor and Cognitive Fatigue.
Performance on the visual memory at baseline correlated significantly with the patient-evaluated patient–therapist agreement on goals: the patients with poorer performance reported a greater agreement. Similarly, patients with poorer performance on attention-executive functions at baseline reported a stronger bond between themselves and the therapist than patients with better performance. No other cognitive variables or self-reported cognitive or fatigue symptoms at baseline significantly correlated with patient-evaluated alliance. Fewer depressive symptoms at baseline correlated significantly with patients' ratings of better patient–therapist agreement on rehabilitation goals.
Neither patient's age or education level correlated significantly with any of the alliance ratings. Nor did alliance ratings differ between men and women. Similarly, duration since MS diagnosis in years did not correlate significantly with any alliance ratings. Nor did patients with milder disability (EDSS 0–4,n = 52) differ from patients with more severe disability (EDSS 4.5–5.5,n = 4) in their alliance ratings.
3.3. Relationship between working alliance and rehabilitation outcome
The correlations between alliance ratings and rehabilitation outcomes are summarised in Table 3 . No change in cognitive outcomes nor self-reported cognitive, depressive or fatigue symptoms or achievement of rehabilitation goals significantly correlated with therapist-evaluated alliance.
|WAI therapist||WAI patient|
|Task a||Bond a||Goal a||Total||Task||Bond||Goal a||Total|
|Change verbal memory||r||0.034||0.079||0.055||− 0.028||− 0.060||0.025||− 0.023||− 0.031|
|Change visual memory||r||0.116||0.170||0.017||0.051||0.102||0.098||0.238||0.129|
|Change attention-executive functions||r||− 0.015||− 0.019||0.015||− 0.069||− 0.171||0.018||− 0.073||− 0.087|
|Change fluency||r||0.010||0.076||− 0.101||− 0.087||0.056||− 0.017||0.166||0.043|
|Change PDQ||r||− 0.153||− 0.083||− 0.086||− 0.098||− 0.059||− 0.087||− 0.101||− 0.062|
|Change MSIS-psyc||r||− 0.219||− 0.153||− 0.225||− 0.133||0.080||− 0.039||0.132||0.079|
|Achievement of personal rehabilitation goals|
|GAS||r||0.114||− 0.021||− 0.027||0.069||0.263||0.154||0.113||0.217|
a Spearman's correlation coefficient; in other variables Pearson's correlation coefficient.
Bold values indicate significance at p < 0.05. Scores were transformed so that a larger value always indicates better performance and positive change.
PDQ = Perceived Deficits Questionnaire; BDI-II = Beck Depression Inventory II; MSIS-psyc = Multiple Sclerosis Impact Scale—psychological dimension; FSMC = Fatigue Scale for Motor and Cognitive Fatigue; FSMC-cog = Fatigue Scale for Motor and Cognitive Fatigue—cognitive; GAS = Goal Attainment Scaling.
No change in cognitive outcomes or patient-reported cognitive or depressive symptoms significantly correlated with patient-evaluated alliance. Patient-evaluated alliance and patient–therapist agreement on the utility and efficacy of the things done in rehabilitation correlated significantly with change in cognitive and overall fatigue symptoms after the intervention. Those patients who evaluated the alliance as better reported fewer symptoms of fatigue after the intervention than those who evaluated the alliance as worse.
Patient-evaluated patient–therapist agreement on the utility and efficacy of the things done in rehabilitation showed a trend towards a significant correlation with patient's achievement of personal rehabilitation goals. Patients who reported agreeing on the utility and efficacy of the things done in rehabilitation with their therapist achieved their goals better.
According to the therapists, 48% of patients benefitted from the intervention moderately, 32% to some extent, and 20% obviously. The therapist's estimate of benefit was associated with therapist-evaluated patient–therapist agreement on rehabilitation goals (WAI goal mean 22.7 (2.4) vs. 23.9 (1.6) vs. 25.0 (1.1),p = 0.030), on the utility and efficacy of the things done in rehabilitation (WAI task mean 21.9 (2.8) vs. 23.2 (2.1) vs. 24.8 (1.4),p = 0.009), and on overall alliance (WAI total score mean 67.8 (7.7) vs. 70.9 (5.3) vs. 74.8 (3.0),p = 0.011). Post-hoc analysis revealed that patient–therapist agreement on the goals (WAI goalp = 0.024) and on the overall alliance (WAI total scorep = 0.008) differed between patients with some and obvious therapist-evaluated benefit from the intervention. Patient–therapist agreement on the utility and efficacy of the things done in rehabilitation (WAI taskp = 0.004), differed between patients with some and obvious therapist-evaluated benefit from the intervention (p = 0.004) as well as between patients with moderate and obvious benefit (p = 0.035). The alliance was better in patients with higher therapist-evaluated benefit. On the other hand, therapist's evaluation of the benefit did not associate with the patient-evaluated alliance.
The aim of this study was to evaluate the effects of the baseline patient- and illness-related factors on the working alliance, as well as the effects of the alliance on rehabilitation outcome in neuropsychological rehabilitation in MS. Baseline factors were not significantly related to alliance. Better patient-evaluated alliance was associated with a more prominent decrease in symptoms of fatigue and better achievement of rehabilitation goals. Better therapist-evaluated alliance was associated with greater benefit from the intervention as evaluated by therapists.
In line with the study of Schönberger et al.  with traumatic brain injury patients with mixed aetiology, we also found that both MS patients and their therapists evaluated the alliance as positive. Moreover, patients rated the alliance more positively than therapists on all scales, which is also in line with previous findings  . The patient-evaluated alliance correlated significantly with that of the therapists, in line with the findings among patients with schizophrenia  .
Baseline patient-related factors did not have a significant effect on the alliance. In line with previous findings among traumatic brain injury patients with mixed aetiology  , we found that the cognitive status of MS patients did not essentially affect the alliance. This indicates that a good alliance is achievable regardless of the severity of cognitive impairments. It has previously been reported that poorer verbal memory and learning performance are related to better alliance in patients with schizophrenia  and traumatic brain injury with mixed aetiology  . We found that poorer performance on visual memory and attention-executive functions was related to better patient-evaluated alliance. Therefore, it could be argued that patients with more severe cognitive difficulties might be in greater need of a strong alliance, or that such patients tend to make more socially favourable appraisals of the working alliance. However, it also must be born in mind that the majority of cognitive domains and self-reports at baseline in the present study showed no relationship to the alliance, and that the present sample represents patients with mild to moderate cognitive problems.
Demographic factors, such as age, gender, education, or disease-related factors, like duration or severity of the disease, did not relate to the alliance. Among patients with traumatic brain injury with mixed aetiology, younger patients tend to report a better alliance  . Our findings are contradictory and do not support the notion that demographic or disease-related factors play a role in the alliance in neuropsychological rehabilitation in MS.
The alliance was not related to rehabilitation outcome in cognitive tests, perceived cognitive deficits, mood, or psychological burden of the disease. Better patient-evaluated alliance was, however, related to a more prominent decrease in fatigue symptoms after the intervention. Additionally, those MS patients who agreed with their therapists on the utility and efficacy of the things done in rehabilitation achieved their goals better. Thus, rehabilitation goals were probably linked to fatigue symptoms, and the methods used in the intervention were also beneficial in their management. Furthermore, patients' satisfaction with the methods used in the intervention related to the actual achievement of the goals.
Greater benefit from intervention as evaluated by therapists was associated with better therapist- but not patient-evaluated alliance. As expected, when therapists found the patient's benefit and success rewarding, they also evaluated the alliance as positive. Partly in line with our findings, therapists' but not patients' experience of a good alliance was found to relate to patients' experience of success in neuropsychological rehabilitation among brain injury patients with mixed aetiology in the study of Schönberger and colleagues  . Difference in evaluations between patients and therapists may partly be explained by the fact that patients and therapists seem to have different views of their alliance, as noted previously  and confirmed in this study.
When interpreting the results, it has to be born in mind, that the alliance evaluated at the end of the intervention might have been confounded by the patient's and the therapist's experience of the rehabilitation result. If rehabilitation is successful, probably also alliance and agreement in the various aspects of the intervention are evaluated as positive. The timing of the outcome assessment and alliance evaluations was near each other but not the same. In more detail, the outcome assessed with neuropsychological tests and questionnaires was not available at the time of alliance evaluations. The end of treatment process data is, additionally, dependent on accurate recollection of impressions accumulated over time. The outcome measures mostly evaluate the status of the patient “at this time”. This is somewhat problematic when using the WAI, which does not provide specific instructions as to the instance or time period the questions are referred. In further studies, repeated alliance evaluations may reduce the risk of method bias and the possibility of the intertwinement between outcome measures and alliance evaluations.
There are limitations in the present study. The major limitation is that the alliance was evaluated only at the end of the intervention. One meta-analysis has reported that the timing of alliance evaluation has not been found to have a significant effect on its relation to outcome  , but also contrast evidence exists in more recent studies and . In several previous studies, , , , , and , the alliance has been evaluated at several time points during the therapeutic process. Therefore, the results of the current study are not completely comparable with previous findings. Several previous studies with neurological diseases have not addressed patients' perspective on the alliance and on outcome, , and , although the quality of the alliance has been reported to be most predictive of outcome when based on patients' evaluations  . In the present study, also the patients' perspective was evaluated. The sample size was quite small, majority of the patients in the current study sample had a relatively mild cognitive decline and patients with more severe disability were under-represented. Therefore, the results are not fully applicable to all patients with relapsing–remitting MS. Furthermore, risk of type I error may be increased because of multiple testing in the present study. Overall the correlations were relatively low and consistent correlations between the subscales were observed. Therefore, caution is needed when interpreting the results of separate subscales. Although the reliability and validity of the outcome measures used in the present study are widely reported, the reliability of the change scores is lower than that of the measures themselves. It cannot be ruled out that this fact could have contributed to the low correlations between change in outcome scores and alliance ratings.
Neuropsychological rehabilitation is more than training and compensation, it is always a process unfolding between people, and the inter-personal process has an intrinsic value. The present study offers preliminary findings on the importance of the alliance in neuropsychological rehabilitation outcome in MS. The significant relations between the working alliance and the benefit from intervention as evaluated by therapists and by goal attainment can be considered as preliminary validation of the WAI in rehabilitation with patients with MS.
To conclude, baseline factors were not found to correlate significantly with the alliance. Further, the patient–therapist alliance may have an impact on the outcome of neuropsychological rehabilitation in MS. Further studies with larger samples and repeated alliance evaluations during the rehabilitation process are needed to verify these preliminary findings.
We thank our patients who kindly agreed to participate in this study. Mervi Rannisto, Mervi Ranta, and Anna Ikonen are warmly thanked for the neuropsychological assessments and Arja Lilja for the neuropsychological rehabilitation given to the patients. Juhani Ruutiainen from the Masku Neurological Rehabilitation Centre and Ilona Autti-Rämö from the Social Insurance Institution of Finland are gratefully acknowledged for their valuable comments on the study design. The study was funded by the Social Insurance Institution of Finland, which is also gratefully acknowledged.
-  C. Fluckiger, A.C. Del Re, B.E. Wampold, D. Symonds, A.O. Horvath. How central is the alliance in psychotherapy? A multilevel longitudinal meta-analysis. J Couns Psychol. 2012;59:10-17 Crossref
-  E. Bordin. The generalizability of the psychoanalytic concept of the working alliance. Psychother Theory Res Pract. 1979;16:252-260 Crossref
-  A.O. Horvath, L.S. Greenberg. Development and validation of the Working Alliance Inventory. J Couns Psychol. 1989;36:223-233 Crossref
-  D.J. Martin, J.P. Garske, K. Davis. Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review. J Consult Clin Psychol. 2000;68:438-450 Crossref
-  A.O. Horvath, A.C. Del Re, C. Fluckiger, D. Symonds. Alliance in individual psychotherapy. Psychotherapy. 2011;48:9-16 Crossref
-  M.D. Leeuw, B.V. Meijel, M. Grypdonck. The quality of the working alliance between chronic psychiatric patients and their case managers: process and outcomes. J Psychiatr Ment Health Nurs. 2012;19:1-7
-  A.M. Hall, P.H. Ferreira, C.G. Maher, J. Latimer, M.L. Ferreira. The influence of the therapist–patient relationship on treatment outcome in physical rehabilitation: a systematic review. Phys Ther. 2010;90:1099-1110 Crossref
-  N.D. Chiaravalloti, J. DeLuca. Cognitive impairment in multiple sclerosis. Lancet Neurol. 2008;7:1139-1151 Crossref
-  D.W. Langdon. Cognition in multiple sclerosis. Curr Opin Neurol. 2011;24:244-249 Crossref
-  A.R. O'Brien, N. Chiaravalloti, Y. Goverover, J. Deluca. Evidenced-based cognitive rehabilitation for persons with multiple sclerosis: a review of the literature. Arch Phys Med Rehabil. 2008;89:761-769 Crossref
-  E.M. Rosti-Otajärvi, P.I. Hämäläinen. Neuropsychological rehabilitation for multiple sclerosis. Cochrane Database Syst Rev. 2011;11:CD009131
-  K.D. Cicerone, D.M. Langenbahn, C. Braden, J.F. Malec, K. Kalmar, M. Fraas, et al. Evidence-based cognitive rehabilitation: updated review of the literature from 2003 through 2008. Arch Phys Med Rehabil. 2011;92:519-530 Crossref
-  P.S. Klonoff, D.G. Lamb, S.W. Henderson. Outcomes from milieu-based neurorehabilitation at up 10 years post-discharge. Brain Inj. 2001;15:413-428 Crossref
-  M. Schönberger, F. Humle, P. Zeeman, T.W. Teasdale. Working alliance and patient compliance in brain injury rehabilitation and their relation to psychosocial outcome. Neuropsychol Rehabil. 2006;16:298-314
-  P.S. Klonoff, M.C. Talley, L.K. Dawson, S.M. Myles, L.M. Watt, J.-A. Gehrels, et al. The relationship of cognitive retraining to neurological patients' work and school status. Brain Inj. 2007;21:1097-1107 Crossref
-  M. Schönberger, F. Humle, T.W. Teasdale. Subjective outcome of brain injury rehabilitation in relation to the therapeutic working alliance, client compliance and awareness. Brain Inj. 2006;20:1271-1282
-  P.S. Klonoff, K.C. Olson, M.C. Talley, K.L. Husk, S.M. Myles, J.-A. Gehrels, et al. The relationship of cognitive retraining to neurological patients' driving status: the role of process variables and compensation training. Brain Inj. 2010;24:63-73 Crossref
-  V. Beckner, L. Vella, I. Howard, D.C. Mohr. Alliance in two telephone-administered treatments: relationship with depression and health outcomes. J Consult Clin Psychol. 2007;75:508-512 Crossref
-  A. Mäntynen, E. Rosti-Otajärvi, K. Koivisto, A. Lilja, H. Huhtala, P. Hämäläinen. Neuropsychological rehabilitation does not improve cognitive performance but reduces perceived cognitive deficits in patients with multiple sclerosis: a randomised, controlled, multi-centre trial. Mult Scler. 2014;20:99-107
-  C.M. Poser, D.W. Paty, L. Scheinberg, W.I. McDonald, F.A. Davis, G.C. Ebers, et al. New diagnostic criteria for multiple sclerosis: guidelines for research protocols. Ann Neurol. 1983;13:227-231 Crossref
-  J.F. Kurtzke. Rating neurologic impairment in multiple sclerosis: an Expanded Disability Status Scale (EDSS). Neurology. 1983;33:1444-1452
-  R.H. Benedict, F. Munschauer, R. Linn, C. Miller, E. Murphy, F. Foley, et al. Screening for multiple sclerosis cognitive impairment using a self-administered 15-item questionnaire. Mult Scler. 2003;9:95-101 Crossref
-  S.M. Rao (Ed.) A manual for the brief repeatable battery of neuropsychological tests in multiple sclerosis (National Multiple Sclerosis Society, New York, NY, 1990)
-  T. Kiresuk, R. Sherman. Goal attainment scaling: a general method of evaluating comprehensive mental health programmes. Community Ment Health J. 1968;4:443-453 Crossref
-  J. Sepulcre, S. Vanotti, G. Hernandez, G. Sandoval, F. Caceres, O. Garcea, et al. Cognitive impairment in patients with multiple sclerosis using the Brief Repeatable Battery-Neuropsychology test. Mult Scler. 2006;12:187-195 Crossref
-  A. Solari, L. Mancuso, A. Motta, L. Mendozzi, C. Serrati. Comparison of two brief neuropsychological batteries in people with multiple sclerosis. Mult Scler. 2002;8:169-176 Crossref
-  J.J.L. Sullivan, K. Edgley, E. Dehoux. A survey of multiple sclerosis: part I: perceived cognitive problems and compensatory strategy used. Can J Rehabil. 1990;4:99-105
-  A.T. Beck, C.H. Ward, M. Mendelson, J. Mock, J. Erbaugh. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561-571 Crossref
-  J. Hobart, D. Lamping, R. Fitzpatrick, A. Riazi, A. Thompson. The Multiple Sclerosis Impact Scale (MSIS-29). A new patient-based outcome measure. Brain. 2001;124:962-973 Crossref
-  I.K. Penner, C. Raselli, M. Stöcklin, K. Opwis, L. Kappos, P. Calabrese. The Fatigue Scale for Motor and Cognitive Functions (FSMC): validation of a new instrument to assess multiple sclerosis-related fatigue. Mult Scler. 2009;15:1509-1517 Crossref
-  R.A. Marrie, D.M. Miller, G.J. Chelune, J.A. Cohen. Validity and reliability of the MSQLI in cognitively impaired patients with multiple sclerosis. Mult Scler. 2003;9:621-626 Crossref
-  O. Gray, G. McDonnell, S. Hawkins. Tried and tested: the psychometric properties of the multiple sclerosis impact scale (MSIS-29) in a population-based study. Mult Scler. 2009;15:75-80 Crossref
-  J. Hurn, I. Kneebone, M. Cropley. Goal setting as an outcome measure: a systematic review. Clin Rehabil. 2006;20:756-772 Crossref
-  T.J. Tracey, A.M. Kokotovic. Factor structure of the Working Alliance Inventory. Psychol Assess J Consult Clin Psychol. 1989;1:207-210 Crossref
-  S. Huolman, P. Hämäläinen, V. Vorobyev, J. Ruutiainen, R. Parkkola, T. Laine, et al. The effects of rivastigmine on processing speed and brain activation in patients with multiple sclerosis and subjective cognitive fatigue. Mult Scler. 2011;17:1351-1361 Crossref
-  M. Schönberger, F. Humle, T.W. Teasdale. The development of the therapeutic working alliance, patients' awareness and their compliance during the process of brain injury rehabilitation. Brain Inj. 2006;20:445-454
-  L.W. Davis, P.H. Lysaker. Neurocognitive correlates of therapeutic alliance in schizophrenia. J Nerv Ment Dis. 2004;192:508-510 Crossref
-  M. Schönberger, F. Humle, T.W. Teasdale. The relationship between clients' cognitive functioning and the therapeutic working alliance in post-acute brain injury rehabilitation. Brain Inj. 2007;21:825-836
-  A.O. Horvath, D. Symonds. Relation between working alliance and outcome in psychotherapy: a meta-analysis. J Couns Psychol. 1991;38:139-149 Crossref
a Tampere University Hospital, Department of Neurology and Rehabilitation, PO Box 2000, 33521 Tampere, Finland
b Seinäjoki Central Hospital, Department of Neurology, Hanneksenrinne 7, 60220 Seinäjoki, Finland
c Tampere University, School of Health Sciences, 33014 Tampere, University Finland
d Masku Neurological Rehabilitation Centre, PO Box 15, 21251 Masku, Finland
© 2013 Elsevier B.V., All rights reserved.