Living systematic reviews

Posted on 20th May 2022 by Philip Heesen

What are living systematic reviews?

Systematic reviews with meta-analyses (SRMA) are performed to systematically analyze primary studies and pool their individual results into overall effect estimates using meta-analytic techniques. They thereby serve as important decision-making tools in clinical practice, and clinical guidelines are often based on the evidence presented in SRMAs.

A limitation of SRMAs is that due to their nature of being very time-consuming to conduct and write, an SRMA is usually only updated every 2 years or more. This means that when new evidence (mostly from randomized controlled trials) arises, this information is not immediately incorporated into an SRMA. Treating a patient according to evidence that might not be up to date can be harmful, but luckily, a new variant of SRMA has been developed and is called a “living systematic review” (LSR).

In an LSR the estimates and summary of findings are continuously updated as new relevant studies are published.

Let’s start with an example of how this new type of SRMA is being/was used. In 2020, following the SARS-CoV-2 outbreak, little has been known about this virus and new information from trials and cohort studies, for example, was emerging every day. Having access to the latest evidence was of the essence and this was achieved by various LSRs answering different research questions ranging from therapy options to prognostic models and more.

When should you consider performing an LSR?

A LSR is not only very time-consuming, but also requires a great amount of resources (multiple collaborators, funding etc.) as you commit to continually updating the review which involves screening for new studies on a very regular basis, selecting relevant studies, extracting data and potentially rerunning meta-analyses. Therefore, LSRs should only be used to answer research questions in fields where new evidence is constantly arising and this new evidence might have a great effect on clinical practice since the certainty of existing evidence is low. However, choosing whether an LSR is a correct study design is up to the discretion of the researchers.

How is an LSR performed?

1. Writing up a protocol: Just like in “regular” SRMAs, a protocol should be written that clearly defines studies that will be included (using the PICO-criteria) and methods that will be used. However, you should also add the frequency of screening for new studies/running literature searches and the timing for incorporation of new evidence from studies into the LSR. Furthermore, special statistical methods might be needed to control for type-I errors (also called ‘false positives’) and other statistical problems, and these methods have to be pre-defined.
2. Initial up-to date review: A first SRMA has to be conducted that summarizes and potentially pools all available relevant evidence. This is the foundation of the LSR since new studies will be added to this initial SRMA and its results and conclusions will be updated.
3. Beginning the actual LSR: After the initial SRMA has been conducted, the work on the actual LSR begins. Now, literature searches must be conducted at pre-specified (see protocol) time-intervals and data from new and relevant studies have to be included in the summary of findings or pooled results from a meta-analysis.
4. Publication: LSRs are usually published on a webpage so that continuous updating of the LSR is immediately visible and accessible to the research and lay community.

Statistical methods

In contrast to “normal” SRMAs, an LSR is frequently updated. This means that the problem of multiple testing and therefore increased type-I error rates and other statistical problems may arise. There are several statistical methods that can be used for continuous updating of LSRs such as trial sequential analysis and modifications thereof. The main goal of these statistical methods are to ensure that the type-I error rate does not exceed a pre-defined level (mostly defined as alpha = 0.05) while simultaneously controlling the type-II error.

Statistical methods for LSRs are very complex and constantly evolving and the interested reader shall be referred to this paper: Living systematic reviews: 3. Statistical methods for updating meta-analyses.

Conclusion

Living systematic reviews are a way of continually updating the results of systematic reviews and meta-analyses and thereby ensuring that the synthesis of evidence is up-to-date. Furthermore, performing a living systematic review is a great and time-consuming endeavour.

References and useful further reading

• Living systematic reviews: 3. Statistical methods for updating meta-analyses
• Living systematic review: 1. Introduction—the why, what, when, and how
• Cochrane. Publishing living evidence
• The COVID-NMA initiative A living mapping and living systematic review of Covid-19 trials
• Drug treatments for covid-19: living systematic review and network meta-analysis
• Update to living systematic review on prediction models for diagnosis and prognosis of covid-19
• Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation
• What is the difference between a systematic review and a meta-analysis?
• Cochrane. Introducing Living Systematic Reviews

Tags: