What do trialists do about participants who are ‘lost to follow-up’?

Posted on 15th December 2020 by Mustafa Thwaini

When undertaking research to assess whether an intervention will have a significant effect on a group of participants, it is essential that all individuals are followed up appropriately so the results of altering the independent variable (intervention) may be analysed.

Let’s consider an example…

In women, how does cutting sugar from their diet affect their weight loss over a period of 6 months?

• Participants: 100 women randomly split into two groups of 50; one experimental and one control
• Intervention: completely cutting the sugar consumption from the participants’ diets in the experimental group
• Control: allowing the participants in the control group to consume sugar as normal
• Outcome measure: the average % change in weight in the two groups is calculated and analysed to see if there is any significant difference
• Timing: participants are followed up every month to measure their weight, then final measurements are taken 6 months into the trial
• Event of interest: weight loss in the experimental group

We can all agree that in order to obtain accurate results of this clinical trial, we must fulfil the adequate information required by the researchers, which, in this case, would be the weight of the individuals at each monthly interval, such that the researchers are able to extensively analyse the results.

When setting up clinical trials, the individuals who are looking to take part in the trial must consent to the implications of the intervention – this consent must be informed and voluntary. Naturally, as humans, those individuals may change their mind and decide to exit the trial prior to having their results collated. Though this may jeopardise the original plans of the researcher, as well as the subsequent research, there may be little the researcher can do to stop this as the participant is not obliged to give a reason for their drop-out. These participants are referred to as lost to follow-up.

This creates the dilemma of what to do with those who have not completed their results; should they be excluded from the study? If so, what should be done with the results that have been collected from them?

Studies which are particularly vulnerable to having participants lost to follow-up are longitudinal studies (those which involve singular or multiple follow-ups after the initial change of the independent variable) e.g., randomised controlled trials (RCTs).

So how do we deal with participants that are lost to follow-up?

Per-protocol

One way to complete statistical analysis where participants are lost to follow-up could be to exclude those lost from the trial itself, such that all the results of the remaining participants may still be analysed. Hence, only those who followed the protocol are included in the trial. This approach is therefore referred to as “per protocol”. It is important to note, however, that this may be an issue within itself, as the problem of attrition bias may arise. For example, removing participants lost to follow-up from an RCT after randomisation (which may be due to the outcome of the study) could act as grounds to externally invalidate the study itself.

If we use our example, one of our participants in the experimental group may have dropped out due to weight gain, even though they’ve been reducing their sugar, resulting in them being disappointed. The researcher excludes them and their monthly weight measurements from the trial. In this case, the researcher feels they are avoiding failure as the trial excludes an individual from the experimental group who failed to achieve the event of interest. This, however, results in bias from the researcher and leads to a reduction in external validation of the study (1).

Intention-to-treat analysis

Another method of accommodating for lost outcome data is through intention-to-treat (ITT) analysis. This involves counting the lost participants into the group they were originally assigned to, even though the data is not complete for that individual. Although the fate of these individuals at the end of the trial would be impossible to know, and hence, the statistical outcome of their results may have some inaccuracy, this mimics real-life situations where patients may decline or be non-compliant to treatments. This would also avoid any bias associated with the per-protocol approach, and hence, some would argue that this provides us with a more valid study. Be that as it may, one must not forget the inaccuracies associated with the results of the study, with the level of inaccuracy depending on the number of participants in the study and the percentage of those who are lost to follow-up (2).

A systematic review of the potential impact on estimated treatment effects of information lost to follow-up in RCTs showed some interesting results. The study concluded that plausible assumptions regarding outcomes in patients lost to follow-up could change the interpretation of the results of trials. The study showed that in the worst-case scenario, where if all those participants of the trials who were lost to follow-up had originated from the experimental groups and had not had the event of interest, the results of 58% of the trials reviewed in the study would have been statistically insignificant. This further reinforces the importance of trialists to appropriately manage their participants who are lost to follow-up, as well as their results, as the way in which the participants are managed may change the direction of the entire study (3).

Conclusions

Although there is absolutely no way to fully prevent losing participants to follow-up, trialists nowadays do their very best to avoid this conundrum. It is imperative however, that if and when participants are lost to follow-up, the way in which those individuals are accounted for in statistical analysis must be accurate and fair, such that the study remains valid, avoids bias, yet retains accuracy.

References (pdf)

You may find these S4BE blogs useful additional reading: 

‘Bias: how much difference does it really make in randomized trials?‘

Statistical significance vs clinical significance

All should be followed up (from the Key Concepts series)

People’s outcomes should be analyzed in their original groups (from the Key Concepts series)

Tags: