Treatments should be compared fairly
Posted on 25th August 2017 by Harriet Pittaway
This is the thirteenth blog in a series of 36 blogs based on a list of ‘Key Concepts’ developed by an Informed Health Choices project team. Each blog will explain one Key Concept that we need to understand to be able to assess treatment claims.
Comparisons are important
Without comparisons, we simply cannot assess whether any given treatment is the “best” for what it is to be used for. If you introduce a treatment without comparing it to something – either the current best treatment for this disease, or – if that is unclear – then a placebo – you run the risk of endorsing a treatment which at best may do nothing, and at worst may cause more harm than good. Without comparisons it is impossible to know if the effect seen is due to a treatment or due to an extraneous factor. Basically, without a comparison you cannot attribute outcomes to a treatment (1, 2).
Even more important, is that a comparison is a fair comparison
Now this is much harder to define and is an area of contention in research as it’s not always easy to tell when a comparison has been fair or not. Making comparisons has probably always been a recognised human trait, frequently seen as “impressions”. For example, a patient may get the impression that they are improving more quickly on a certain treatment than they had done on a previous one. But it would be unwise of their doctor to extrapolate this information and switch all of their patients to this other treatment. Therefore, we need robust comparisons.
When looking to bring a single treatment onto market for use by doctors and patients, the most widely regarded gold standard of comparing this to an alternative is to complete randomized controlled trials (RCTs). RCTs are good because they aim to create comparison groups that are similar except for the treatments received by each. Ideally, multiple RCTs would be conducted and then synthesized in a systematic review.
Let’s take a hypothetical example
Imagine a drug company designs a new pill to reduce blood pressure. You want to assess whether it works as well as you hope it does. Do you choose to compare your new drug to a placebo (a sugar pill that makes no physiological change to the participant) or to the current best treatment for your patient group? Now this is a choice which has huge implications for the results.
When claims such as ‘new blood pressure tablet reduces your risk of 
complications by 20%’ are made in public, a very important question needs to be asked: is this 20% reduction when compared against placebo or when compared again the current best treatment?
If it’s 20% compared to placebo, but the current gold standard reduces your risk by 25% compared to placebo, I know which one I’d rather give to my patients.
You might think that this sort of practice is not that common though, because it seems a little misleading. You’d be wrong. In a podcast with the Guardian in 2012, Ben Goldacre points out that 1/3 of the drugs which came to market through the Food and Drugs Administration in the United States between 2000 and 2010 were compared with placebo despite effective drugs currently being used for the disease that the research teams were targeting (3). This doesn’t make this research completely redundant but it certainly highlights how important it is that health professionals, researchers and patients are vigilant about what the comparison is that is being made.
‘Comparison’ is defined in the Oxford dictionary as ‘a consideration or estimate of the similarities or dissimilarities between two things or people’ (4). I hope this blog has helped you to see that in healthcare this isn’t really enough. A consideration or estimate is dangerous for patients when it comes to treatments, we need fair and appropriate comparisons.
My take home messages are:
- Without a comparison, you cannot know whether a treatment is worth using because it’s not possible to know what would happen without the treatment.
- Comparisons can be made informally as impressions, but they need to be explored in well-designed trials to produce strong scientific evidence to be valuable to clinicians and their patients.
- Giving or promoting treatments that have not been robustly shown to be effective through fair comparisons can be dangerous.
