Conducting a systematic literature search

Posted on 18th July 2017 by Saul Crandon

Systematic reviews sit amongst the top of the evidence hierarchy. This is all well and good, provided they are conducted appropriately. To ensure a review can provide high quality, reliable evidence, they must be completed meticulously, following reporting guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the Cochrane Handbook (1, 2) A key element of this is a robust search strategy.

Systematic reviews take existing literature and synthesise it, either qualitatively or quantitatively. In order to be ‘systematic’ and minimise selection bias, the search must be comprehensive, encompassing all of the relevant research. Bias will exist if the authors fail to include ALL of the pertinent research. “How can you be sure your conclusions are true if you have not explored all of the evidence?”

Of course, some articles may be excluded (for instance if they are methodologically flawed) once you come to screening the evidence, especially if this is a meta-analysis. However, the authors must ensure that initially all of the relevant evidence is captured by the search.

Performing an in-depth search is often overwhelming for authors setting out with a systematic review. It can seem confusing or difficult and is often substituted for a more manageable, less rigorous search. The following checklist should provide a clear framework for those wanting to ensure their search is truly systematic.

All illustrative example of database searching will be performed on Ovid Medline. Please note that this is a basic worked example to illustrate some of the key principles involved in searching. Using the search outlined below, you are likely to pick up key studies. However, to minimise your chances of missing relevant articles then do consultant a librarian who will be able to assist you with advanced search techniques. This is the best way to maximise sensitivity and reduce your risk of missing relevant articles.

BEFORE THE SEARCH

Firstly, it is important that systematic review protocols (including search strategies) are prospectively registered to prevent deviation from the intended methods and minimise bias. A popular registration database is the International Prospective Register of Systematic Reviews (PROSPERO).

Also, it is a good idea to carry out a search for any similar systematic reviews. It may be pointless to repeat an existing systematic review. Although if an existing systematic review is outdated or features flawed methodology, then a new systematic review on that topic can be justified.

Defining the research question

Before a search can be performed, it is crucial that the research question is explicitly defined. There are many ways to do this, but the most common method is to divide your question according to the Patients, Interventions, Comparisons and Outcomes (PICO) model.

Patients: Which patient population do you want to explore?
Adults? Elderly? Paediatrics? Males? Females? Certain ethnicities? Inpatients? Community patients? Certain co-morbidities?

Interventions: What intervention(s) do you want to explore?
A certain medication? A certain surgical procedure?

Comparisons: What are you going to compare the intervention(s) against? (Some reviews may not have a comparison so this section is optional).
Are you going to compare the intervention with no intervention? A control? Usual treatment? A different medication? A different surgical procedure?

Outcomes: What outcomes do you want to explore?
Mortality rate? Morbidity or the development of a certain disease? Change in baseline parameters such as blood pressure, weight or cholesterol?

Defining the parameters of the search

Once a well-defined research question has been established, it is important to outline where you will search for the evidence. Systematic searches should aim to search as many different sources as possible. This can be broken down into the following:

Sources
Online databases* (this will be the major area for medical literature searches)
Books/physical literature
Grey literature* (this refers to unpublished material/published in a non-commercial form)
Ongoing trials (clinicaltrials.gov)

Types of articles
Randomized controlled trials (RCTs)
Cohort studies
Case-control studies
Case series studies
Case reports
Other

Dates
How far back do you wish to explore?
Perhaps the intervention you want to investigate was only developed in the last 10 years

Languages
English
Other languages too? Ideally you should be willing to translate these in order to fully understand their findings

Subjects
Humans
Animals
Other

*The following databases are useful for clinical systematic reviews:

PsychINFO – key database for mental health literature
MEDLINE – large medical database
EMBASE – large medical database
SCOPUS – includes many scientific disciplines
Cochrane Library – high-quality evidence
Web of Science – includes many scientific disciplines
CINAHL – includes biomedicine, healthcare, nursing and allied health articles

*Examples of grey literature databases include:
OpenGrey
Copac

THE SEARCH ITSELF

Searching online databases

When searching online databases, the terms and their synonyms for each of the components of the PICO model must be written out, including abbreviations. It is also important to use alternate spellings and word endings. This can be done using a number of strategies within the database:

Firstly, truncation involves putting an asterisk (*) in a word with a variable ending e.g. toxic* will search for toxic, toxicity and toxicology.

Secondly, using a question mark (?) in place of a single letter for words that have alternate spellings (such as American vs British English) will search for both spellings, e.g. p?ediatric will search for both paediatric and pediatric. The question mark replaces a single character or no character, allowing you to pick up alternate spellings. It should be noted that these terms can vary between databases, and so should always be checked prior to conducting a search.

You should also search for hyphenated terms with and without the hyphen as different authors may title their work differently.

An example to illustrate a search in Ovid Medline:

Let’s imagine we want to conduct a systematic review to assess the effect of primary percutaneous coronary intervention (PCI) versus streptokinase on mortality in myocardial infarction (MI) patients. (Clicking on the screenshots below will enlarge them). 

Patients: myocardial infarction, MI, myocardial necrosis, heart attack

Interventions: primary percutaneous coronary intervention, PCI, percutaneous coronary intervention, primary PCI, reperfusion, re-perfusion, angioplasty 

Comparisons: streptokinase, medical reperfusion, medical re-perfusion

Outcomes: mortality, death, survival

As you can see, each line represents a synonymous term that is searched for, with the corresponding number of articles found in the results column. These terms can be combined using ‘OR’ to ensure the database captures all of the articles relating to these terms.

For the final screenshot, all of the groups of terms are combined with ‘AND’ to ensure the database only displays results that are related to the combination of terms (i.e. about PCI and streptokinase in MI patients). In this example, it yielded a total of 254 results.

Further refining or increasing your results

Medical Subject Headings (or MeSH terms) are terms predefined by the database using human indexers in concordance with thorough protocols. MeSH terms encompass ‘Headings’, ‘Subheadings’, ‘Supplementary Concept Records’ and ‘Publication Characteristics’. Further definitions of these categories and relevant examples can be found on the National Library of Medicine’s website (https://www.nlm.nih.gov/bsd/disted/meshtutorial/introduction/04.html). The same website also provides detailed descriptions of how the MeSH system is structured and how this can be searched to further refine or expand your literature search. Using Medline as an example, you must select ‘Map term to subject heading’ to search for MeSH terms.’

If your search yields no results then it is important to re-evaluate your PICO model and broaden the search terms. Conversely, if the results are very expansive this will take considerable effort to screen the relevant information. It may be worth narrowing the search terms in order to achieve a more focused search.

AFTER THE SEARCH

Any more studies to include?

Once the formal search has been completed, you must now remove the duplicates and screen the titles/abstracts of the remaining results. This is a large task in itself, and information on how this can be done is beyond the scope of this blog.

Once competed, a further exploration for relevant studies can begin. There are a number of strategies that should be carried out at this stage.

• Citation tracking – This involves searching for studies that have referenced the included studies in their work, and as such, may be relevant to your systematic review. Most databases will have an integrated tool to perform this.
• Manual reference searching – Once you have finalised a list of applicable studies, you should go through their references individually and search for additional relevant studies.
• Contacting authors – Finally, authors of the included studies can be contacted to search for further results that may be pertinent to your review, or to ensure your search strategy has not missed any of their other work which may be useful.

At this point it is crucial that you finalise your list of included studies and document your search well so that you can refer to this in the future and during the rest of the review process.

Your SYSTEMATIC literature search is now complete!

Congratulations! If you have followed the various steps in this article, you are well on your way to completing a comprehensive systematic review! If you need further help, consult your librarian.

References

1. Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from cochrane-handbook.org.
2. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses:The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097

Tags: