Stroke rehabilitation trials: let’s measure what matters
Posted on 13th March 2018 by Julie Duncan Millar
In this blog, Julie Duncan Millar, PhD Student and Physiotherapist, reflects on the difficulties of comparing and sharing upper limb rehabilitation trial data and proposes a condensed toolkit of measures recommended for researchers to use in future trials. The toolkit was developed by considering the stroke survivor, family and friend, clinician and researcher perspectives on life after stroke with upper limb dysfunction.
As an NHS specialist neurological outpatients physiotherapist, I worked regularly with stroke survivors and their family and friends. Up to three quarters of stroke survivors have some form of dysfunction affecting the arm, shoulder and/or hand. So it was no surprise that they would commonly ask for help with their upper limb: the grandparent who wanted to get back to looking after their grandchildren involving lots of cuddling, lifting, and playing; the parent who needed to get back to driving to allow them to drop the kids at school before heading to work; the keen golfer wanting to get back to swinging a club and playing with friends. I was always striving to do the best by my patients by providing a person centred approach to rehabilitation. I’d keep asking myself: Which intervention (treatment) is most appropriate for this person, at this time, at this stage in their recovery? I considered the evidence: What is the most effective dose of the intervention? How much work may this person have to put in to the treatment and for what gain? How much will it improve my patient’s life?
Person-centred stroke rehabilitation. Which treatment for this person, at this time? How much work will they have to put in and how much will it improve their life?
Upper limb rehabilitation: what the evidence says
Keen to keep up to date, off I trotted to find the latest evidence on upper limb rehabilitation following stroke. There was a lot of data. So much data in fact that a group of editors from Cochrane Stroke Group, led by Dr. Alex Pollock, had completed a Cochrane overview (a systematic review of systematic reviews) into ‘Interventions for improving upper limb function after stroke’. “Hurrah”, I thought, “All of my questions, finally answered”. Well…sort of.
After pooling data from 40 reviews (19 Cochrane Reviews and 21 non-Cochrane) on 503 studies on over 18,000 participants, and lots of complex number crunching, they concluded that…drum roll please… there was no high quality data to support the use of any of the interventions in the overview. None. Zero. Zilch. That meant I couldn’t say with certainty which of the upper limb interventions in my clinical repertoire might work best for each of my patients.
However, all was not lost. There was evidence that some interventions would probably have a beneficial effect: constraint-induced movement therapy, mental practice, mirror therapy, interventions for sensory impairment, virtual reality and relatively high dose of repetitive task practice, as well as an indication that unilateral upper limb training may be more effective than bilateral upper limb training. Yet, it was difficult to identify a suitable dose for these interventions.
While these results were really useful there was nothing saying this is likely to work, for this type of patient with this type of problem with their upper limb. That wasn’t the reviewer’s fault, the data just weren’t available.
The problem of too many outcomes and a proposed solution
One way to tackle this problem is to look at the comparability of the data produced in trials investigating the interventions. When undertaking the overview, Dr. Pollock and her team had to pool together data from 503 different studies to identify the most effective interventions. That’s a lot of data; a lot of outcomes to compare, that were captured by a lot of different outcome measures. In fact, at least 48 different upper limb outcome measures have been used in upper limb rehabilitation trials (Santisteban et al., 2016). That is only considering the ‘typical’ upper limb outcomes such as grip strength, pain in the upper limb, or ability to dress self using both arms. That doesn’t even start taking into consideration outcomes beyond these ‘typical’ ones. Yet we know that it’s not just these ‘typical’ outcomes that affect stroke survivors with upper limb dysfunction. For example recent qualitative work identified that life after stroke and sense of self are also affected (Purton 2017) so, from a stroke survivors perspective, these may also need consideration.
Unsurprisingly, researchers realise that using all of these different outcome measures makes it difficult to share and compare data to identify effective treatments. Therefore, recently the Stroke Rehabilitation and Recovery Roundtable (SRRR) agreed consensus recommendations on the measures researchers should use in all sensorimotor trials (Kwakkel et al., 2017). It’s great to see a drive towards consensus on outcome measure use in trials and I’m well and truly on board. So too are clinicians as demonstrated by the standard set of outcome measures recommended for stroke clinicians by ICHOM. Owing to the complexity of the upper limb, I believe we need to build on the momentum of the SRRR and support informed selection of the measures used specifically in upper limb rehabilitation trials. Herein followed my PhD: Standardising Measures in Arm Rehabilitation Trials (SMART).
Measuring what matters to stroke survivors
Initiatives such as COMET are here to support standardisation work, but at present there is no gold standard way to agree on which outcome measures to use in trials. However, a central point to all outcome measure recommendations is to consider the perspective of those living with the condition. Fantastic. That makes perfect sense. In order to find out if an intervention has a meaningful effect on important outcomes, you need to identify important outcomes by speaking to those who know most about the condition.
What matters most about the arm and how it affects life after stroke?
I took the angle that if you are a researcher about to conduct a trial, or pool data in a review, I believe you would want to capture outcomes that are deemed important by those who will ultimately use, and benefit from, your research findings: stroke survivors, family and friends, and clinicians. I think we all agree that capturing outcomes using valid and reliable measures is important, but I argue that if the outcome you are measuring has no relation to life after stroke, what’s the point?
A key phase of my PhD involved identifying important outcomes by asking stroke survivors, family and friends, and clinicians ‘What matters most about the arm and how it affects life after stroke?’. The results of this study informed the remainder of my PhD: I linked these outcomes to all of the different outcome measures used in trials within the Cochrane Overview (Pollock et al., 2014) to see which outcome measures captured which important outcomes. I then conducted an international two round online consensus survey, called an eDelphi, with researchers and clinicians to identify the most important and feasible measures. Finally, I held an international consensus meeting with all stakeholders to agree the measures that would be recommended for use in all future stroke upper limb rehabilitation trials.
When I talk to people outwith research about my PhD I am often met with a puzzled look: ‘Don’t researchers already know what to measure in trials?’ It seemed sort of obvious. Then I realised, researchers don’t always have the luxury I had as a clinician. I could simply ask my patient ‘What matters most to you?’ I could easily make my treatment ‘person centred’. Researchers couldn’t easily make their trials ‘person centred’ because it wasn’t completely clear what outcomes are important to stroke survivors, family and friends, and clinicians.
The SMART toolbox
So in order to bridge this gap and support informed selection of outcome measure use in future upper limb rehabilitation trials, I have developed the SMART toolbox. This was the end result of my PhD. As detailed above, it involved a great deal of engagement with stroke survivors, family and friends, clinicians and researchers to develop these international consensus recommendations. For researchers designing future trials or conducting reviews on upper limb rehabilitation, they simply need to look in the SMART toolbox and select their primary and secondary outcome measures across the International Classification of Functioning, Disability and Health framework, while considering their own individual trials aim. Of course, researchers are free to select other measures from outwith the SMART toolbox, but we would recommend starting at the SMART toolbox as the ‘go to’. Not only do we expect this to improve data comparability but also researchers will know when selecting the measures that they are capturing outcomes that matter to stroke survivors, family and friends, and clinicians. No more endless searching and debating the most important measures; the hard work has already been done. Therefore selecting measures from the SMART toolbox will also reduce study set up time.
We accept that, like most things in life, there are limitations and the SMART toolbox isn’t perfect; but I don’t think a perfect measure exists. However careful selection from these agreed measures is a pragmatic solution and I believe that the main strength of my project is the extensive stakeholder engagement in the consensus process. The effects of stroke are multifaceted, the arm is complicated, and life after stroke is unique for each person. There are lots of things to take into consideration when selecting outcome measures to use in trials or to pool in reviews. However I believe that by involving stroke survivors, family and friends, and clinicians in this study, we are a step closer to making things simpler for researchers, who in turn make it easier to identify effective interventions for clinicians. This will ultimately improve life after stroke for stroke survivors. To learn more, and start using the SMART toolbox, keep your eyes peeled for our upcoming papers.
I would like to acknowledge the guidance and input from my ever supportive supervisory team – Dr. Myzoon Ali, Prof. Frederike van Wijck and Dr. Alex Pollock.
Join in the conversation on Twitter with @jduncanmillar @CochraneUK #LifeAfterStroke or leave a comment on the blog.
Keep up to date with the SMART toolbox: on Research gate, NMAHP RU or Twitter
References may be found here
Julie Duncan Millar has nothing to disclose.
