An update to the Evidence-Based Medicine Pyramid: why and how
Posted on 4th August 2017 by Cole Wayant
When it comes to pyramids, they’re not all made alike.
The Great Pyramids in Egypt may continue to stand strong for a millennium to come, but the Evidence-Based Medicine Pyramid may have only lasted a few decades. A complete and charming tour of the Evidence-Based Medicine Pyramid and its many components can be read here, and it is highly recommended.
In short, the Evidence-Based Medicine Pyramid is comprised of many levels of evidence, with levels closer to the peak representing types of evidence known to be more rigorous and reliable.
Systematic reviews and meta-analyses have made their home at the peak for many years, and the Cochrane Collaboration has famously demonstrated the necessity of systematic reviews and meta-analyses in such topics as the benefits of Tamiflu® for influenza (1) and corticosteroids in pregnant women about to give birth prematurely (2).
In August 2016, however, a Perspective Article was published in Evidence Based Medicine, a journal owned by the British Medical Journal, that argued for a restructuring of the Evidence Pyramid to allow more fluidity in the appraisal of evidence across different study types (3).
The old Evidence Pyramid was too simplistic and rigid, the authors argue, so they proposed two modifications:
Modification number 1:
According to the old Pyramid, a randomized controlled trial, where two or more groups are randomized to either the intervention (new drug or therapy being tested) or the control (placebo or current standard of care), was always superior to observational study. The most poorly conducted randomized trial would technically still be weighted higher on the Pyramid than the most methodologically sound observational study. But this is not always the case in practice!
Citing observational studies of hip replacements in patients with osteoarthritis, where no randomized trials have been conducted to date, the authors of the Perspective state that we should be able to look at these well-conducted observational studies as evidence of the hip replacement’s benefit, despite the lack of “higher levels of evidence”. So, the authors recommend a modification to the rigid, straight lines separating levels of evidence and recommend the adoption of wavy lines. The wavy lines demonstrate the ability for study types lower on the pyramid to outshine and outweigh the types of evidence that are typically higher in evidence-based clinical practice.
Modification number 2:
Secondly, the authors recommend removing systematic review and meta-analysis from the top of the pyramid since not all meta-analyses are created equal. The meta-analysis, often depicted as a forest plot (Fig 1), can be a key part of a systematic review, and demonstrates the overall treatment effect of an intervention (drug, therapy, etc.). But what if there is a meta-analysis of studies whose design places it low on the Pyramid? Is a meta-analysis of case reports, small group accounts of an outcome, to be compared to a meta-analysis of double-blind, placebo-controlled randomized controlled trials? For this reason, removing meta-analysis from the top of the pyramid makes sense, because in many cases a meta-analysis of case reports may present less robust data than a single well-conducted randomized trial.
Figure 1. Presented in S4BE’s previous blog “Tutorial: How to read a forest plot”. Image adapted from Table 4 Roberts et al. (2006)2.
But wait, there’s more…
If systematic reviews and meta-analyses are so important, what are we to do if we remove them from the Evidence Pyramid? Enter the most compelling modification purported by the authors. We use systematic review and meta-analysis as a lens through which we are able to clearly see and understand the quality of evidence being published at each level of the Evidence Pyramid. And in practice, this makes sense.
An obstetrician, after having read a systematic review about the benefits to the child of giving mothers corticosteroids during premature labor, is more likely to adopt a newly published, more precise treatment regimen in light of the already established benefit of corticosteroids. Creators of clinical practice guidelines will be able to incorporate well-conducted observational studies into highly graded recommendations, instead of relegating all recommendations based on observational studies alone to medium or low grades.
The restructuring of the Evidence Pyramid seems likely to grant more freedom and confidence to physicians, who can interpret and appraise evidence as it is published, rather than waiting years for a new or updated systematic review.
So there you have it. Unfortunately, physicians may not be able to boast for generations to come of the invulnerability of their Evidence-Based Medicine Pyramid, but if patients achieve better quality outcomes, no one will think to care.
1 Jefferson T, Jones MA, Doshi P, Del Mar CB, Hama R, Thompson MJ, Spencer EA, Onakpoya IJ, Mahtani KR, Nunan D, Howick J, Heneghan CJ. Neuraminidase inhibitors for preventing and treating influenza in adults and children. Cochrane Database of Systematic Reviews 2014, Issue 4. Art. No.: CD008965. DOI: 10.1002/14651858.CD008965.pub4.
2 Roberts D, Brown J, Medley N, Dalziel SR. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database of Systematic Reviews 2017, Issue 3. Art. No.: CD004454. DOI: 10.1002/14651858.CD004454.pub3.
3 Murad MH, Asi N, Alsawas M, Alahdab, F. New evidence pyramid. Evidence-Based Medicine. 2016;21:125-127
No Comments on An update to the Evidence-Based Medicine Pyramid: why and how
“The most poorly conducted randomized trial would technically still be weighted higher on the Pyramid than the most methodologically sound observational study.”
No, a poor quality RCT would not even get into the pyramid.
“The meta-analysis…. is a crucial part of a systematic review…”
Are they not different things?
“…a meta-analysis of case reports may present less robust data than a single well-conducted randomized trial.”
A truism. Does anyone apart from quacks seriously meta-analyse case reports?9th August 2017 at 10:58 am
I would also urge caution about thinking doctors (or even guideline developers) behave rationally. Sadly obstetricians didn’t act for many years on the strong systematic reviews of evidence about the benefits of corticosteroids….8th August 2017 at 11:13 pm
Best wishes Susan Bewley
I enjoyed reading your blog post. One element on which I would urge caution is your statement that “The meta-analysis, often depicted as a forest plot (Fig 1), is a crucial part of a systematic review.” A systematic review sets out to reduce biases in assembling and interpreting relevant research. Meta-analysis (statistical synthesis of research results) is not a necessary feature of a systematic review: it may be impossible or, even if its possible, it may be inappropriate. This may be of interest: http://www.jameslindlibrary.org/chalmers-i-altman-dg-1995/.
Best wishes, Iain C4th August 2017 at 1:35 pm