People’s outcomes should be analyzed in their original groups
Posted on 29th September 2017 by Sam Marks
This is the fifteenth blog in a series of 36 blogs based on a list of ‘Key Concepts’ developed by an Informed Health Choices project team. Each blog will explain one Key Concept that we need to understand to be able to assess treatment claims.
Imagine you’re reading a study and you see that the participants were randomly allocated to two treatment groups. One group is allocated to receive surgery and the other to receive a drug intervention. Great, you think, they’ve been randomly allocated: this helps ensure the groups have similar characteristics. But it doesn’t stop there. For various reasons, participants may not actually receive or take their allocated treatment. A participant may choose not to take the treatment for example.
It’s important that – as far as possible – all the patients allocated to these two groups are followed up and included in the main analysis of the group to which they were allocated. Even if – and this might initially seem counterintuitive – they never actually received the intervention they were allocated. This is because the characteristics of those who do not receive or take their treatment may differ from those who do. Consequently, excluding those who did not take their allocated treatment from the analysis may mean that your comparison is no longer fair. That is, you are no longer comparing like with like.
Including participants in the analysis irrespective of whether they actually receive their allocated treatment is known as ‘intention-to-treat’ (ITT) analysis
Fisher et al. (1990) explain: “The ITT analysis includes all randomized patients in the groups to which they were randomly assigned, regardless of their adherence with the entry criteria, regardless of the treatment they actually received, and regardless of subsequent withdrawal from treatment or deviation from the protocol”
Intention-to-treat analysis is usually described as “once randomized, always analyzed”
For instance, in a comparison of surgery and drug treatments, people who die while waiting for their surgery should be counted in the surgery group, even though they didn’t actually receive the surgery. If they are not, you’re not actually comparing like with like.
Intention-to-treat analysis avoids overestimating the efficacy and/or safety of an intervention which could result from ignoring those who never receive the treatment. This is because it accepts that non-compliance and non-adherence are likely to occur in actual clinical practice. That is, the effectiveness of a treatment is not simply determined by its actual biological effects. It’s also influenced by the patient’s ability to adhere to, and/or the clinician’s ability to administer, the intended treatment. Only by retaining all patients intended to receive a given treatment in their original treatment group can researchers and practitioners obtain an unbiased estimate of the effect of selecting one treatment over another.
Let’s take an example
Imagine you’re comparing two intensities of cancer treatments (a high intensity and a low intensity). In the high intensity group, 10 of 50 participants choose not to take their treatment due to the unpleasant side effects of the treatment. In the low intensity group, the side effects are less extreme and only 3 of 50 participants choose not to take their treatment. Imagine then if you were only comparing the 40 patients who continued to take the high intensity treatment with the 47 who continued to take the treatment in the low intensity group. This would be misleading, because you’d be failing to take into account the tolerability of the treatment, when the tolerability of a treatment is very important factor!
Let’s take another example
The figure, and example below, help to further illustrate why it’s important to carry out intention-to-treat analysis:
Example and illustration taken from Testing Treatments 2nd edition. Available from: http://bit.ly/2hyvb4g
Patients with partial blockages of blood vessels supplying the brain and who experience dizziness have an above average risk of having a stroke. Researchers have investigated whether operating on these patients to unclog blood vessels would reduce subsequent strokes.
They compared all patients allocated to the operation, regardless of whether or not they survived the surgery, with all those patients in the comparison group, who did not receive surgery. If they’d recorded the frequency of stroke only among those patients who’d survived the immediate effects of the surgery, they would have missed the important fact that the surgery itself can cause stroke and death. So – if all other things were equal – the surviving participants in this group would have fewer strokes. That would be an unfair comparison because you need to consider the risks of the surgery itself.
As shown in the figure above, the outcomes (i.e. mortality rate) of the surgery vs. the medical treatment are actually equal. But if the 2 people allocated to the surgery who had died before they actually received surgery were ignored (i.e. excluded from the analysis), the comparison of the 2 groups would have been biased. The results would indicate that surgery is superior to the comparator treatment when actually this is not the case.
So what does this all mean for us, when we’re reading a study or a review?
Unfortunately, researchers are often bad at applying intention-to-treat analysis and/or journals often poorly report on whether this analysis has been carried out. But it’s really important to be wary of, and cautious about, relying on the results of treatment comparisons if participants’ outcomes are not counted in the group to which they were allocated. Remember, best practice is: “once randomized, always analyzed”.
