Are the treatments practical in your setting?
Posted on 15th May 2018 by Ed Walsh
This is the thirty-third blog in a series of 36 blogs based on a list of ‘Key Concepts’ developed by an Informed Health Choices project team. Each blog will explain one Key Concept that we need to understand to be able to assess treatment claims.
When looking at research, people often regard the results as the important part. Indeed, they are important. You shouldn’t use a new treatment unless results show it can work. The results show the treatment works! Now you can use it, right?
Not so fast. ‘Wise men say only fools rush in’ – the king himself (Elvis) making a point especially relevant when interpreting results. To get to a result you have to do things a certain way. If you can do things the way the research did, no problem! But if you can’t do things like the research did, then you may not be able to get the same results!
Think about making a cake. You’ve seen Mary Berry make a wonderful cake on TV and everyone says it tastes amazing. You go in to your kitchen ready to make an equally wonderful cake. You start getting the ingredients together and realise you have no eggs. Not an egg-cellent start. You crack on anyway and things get worse. You’re quite clumsy and you end up spilling half the cake mix on the floor. Egg-sactly what you didn’t want to happen. You put what’s left of your creation in the oven, but you only have 20 minutes before you’re supposed to leave for your eggs-ercise group. Mary baked her cake for 30 minutes. You take your cake out. Unsurprisingly, it looks and tastes nothing like Mary Berry’s wonderful cake. Mary Berry got better results because she had the ingredients, skill and time to achieve them. You got your cake because lacked those things.
The same applies to research. If the equipment, skill level and time used to achieve results in research is better than the equipment, skill level and time that you have available, you won’t get the same results. That’s why the methods are so important!
Heart Surgery
How do these principles apply to treatment? Surgery is a good example. A new form of heart surgery performed in a state of the art, specialised hospital may show very positive results. Undertaking the new surgery in the specialised hospital was the best heart surgeon in the world, internationally renowned for her superb dexterity. All the best surgical equipment was available to her, and she was given as much time as she needed to get the best possible results.
The same new heart operation may be performed in other hospitals, but the results may be different. In other hospitals, the difference may arise because the surgeon is less experienced. The equipment available in non-specialist hospitals may also be different. Or maybe the surgeon has less time to perform the surgery in different hospitals. All of these factors are reasons why the same surgery can have different outcomes in different settings. It’s not just surgery that this applies to.
‘Inflamaway’
Imagine a new study finds that inflamaway, a new (hypothetical) anti-inflammatory drug, is more effective at reducing inflammation than ibuprofen. Ibuprofen is commonly used to treat inflammation in your setting, so should everyone use inflamaway instead of ibuprofen?
You notice that the dosage of ibuprofen used in the study is much lower than the dosage used in your setting. The low dose of ibuprofen in the trial may be less effective at treating inflammation than the higher dose used in your setting. Because inflamaway hasn’t been compared to the dosage of ibuprofen used in your setting, you don’t know if inflamaway is actually better in your setting.
Hopefully these examples make clear that if your circumstances are quite different from those in the research studies which have tested a treatment, the results may not apply to your setting. Results are important, but they mean very little without knowing about the methods for achieving them.
