Conducting a systematic literature search
Posted on 18th July 2017 by Saul Crandon
Systematic reviews sit amongst the top of the evidence hierarchy. This is all well and good, provided they are conducted appropriately. To ensure a review can provide high quality, reliable evidence, they must be completed meticulously, following reporting guidelines such as the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the Cochrane Handbook (1, 2) A key element of this is a robust search strategy.
Systematic reviews take existing literature and synthesise it, either qualitatively or quantitatively. In order to be ‘systematic’ and minimise selection bias, the search must be comprehensive, encompassing all of the relevant research. Bias will exist if the authors fail to include ALL of the pertinent research. “How can you be sure your conclusions are true if you have not explored all of the evidence?”
Of course, some articles may be excluded (for instance if they are methodologically flawed) once you come to screening the evidence, especially if this is a meta-analysis. However, the authors must ensure that initially all of the relevant evidence is captured by the search.
Performing an in-depth search is often overwhelming for authors setting out with a systematic review. It can seem confusing or difficult and is often substituted for a more manageable, less rigorous search. The following checklist should provide a clear framework for those wanting to ensure their search is truly systematic.
All illustrative example of database searching will be performed on Ovid Medline. Please note that this is a basic worked example to illustrate some of the key principles involved in searching. Using the search outlined below, you are likely to pick up key studies. However, to minimise your chances of missing relevant articles then do consultant a librarian who will be able to assist you with advanced search techniques. This is the best way to maximise sensitivity and reduce your risk of missing relevant articles.
BEFORE THE SEARCH
Firstly, it is important that systematic review protocols (including search strategies) are prospectively registered to prevent deviation from the intended methods and minimise bias. A popular registration database is the International Prospective Register of Systematic Reviews (PROSPERO).
Also, it is a good idea to carry out a search for any similar systematic reviews. It may be pointless to repeat an existing systematic review. Although if an existing systematic review is outdated or features flawed methodology, then a new systematic review on that topic can be justified.
Defining the research question
Before a search can be performed, it is crucial that the research question is explicitly defined. There are many ways to do this, but the most common method is to divide your question according to the Patients, Interventions, Comparisons and Outcomes (PICO) model.
Patients: Which patient population do you want to explore?
Adults? Elderly? Paediatrics? Males? Females? Certain ethnicities? Inpatients? Community patients? Certain co-morbidities?
Interventions: What intervention(s) do you want to explore?
A certain medication? A certain surgical procedure?
Comparisons: What are you going to compare the intervention(s) against? (Some reviews may not have a comparison so this section is optional).
Are you going to compare the intervention with no intervention? A control? Usual treatment? A different medication? A different surgical procedure?
Outcomes: What outcomes do you want to explore?
Mortality rate? Morbidity or the development of a certain disease? Change in baseline parameters such as blood pressure, weight or cholesterol?
Defining the parameters of the search
Once a well-defined research question has been established, it is important to outline where you will search for the evidence. Systematic searches should aim to search as many different sources as possible. This can be broken down into the following:
Online databases* (this will be the major area for medical literature searches)
Grey literature* (this refers to unpublished material/published in a non-commercial form)
Ongoing trials (clinicaltrials.gov)
Types of articles
Randomized controlled trials (RCTs)
Case series studies
How far back do you wish to explore?
Perhaps the intervention you want to investigate was only developed in the last 10 years
Other languages too? Ideally you should be willing to translate these in order to fully understand their findings
*The following databases are useful for clinical systematic reviews:
PsychINFO – key database for mental health literature
MEDLINE – large medical database
EMBASE – large medical database
SCOPUS – includes many scientific disciplines
Cochrane Library – high-quality evidence
Web of Science – includes many scientific disciplines
CINAHL – includes biomedicine, healthcare, nursing and allied health articles
*Examples of grey literature databases include:
THE SEARCH ITSELF
Searching online databases
When searching online databases, the terms and their synonyms for each of the components of the PICO model must be written out, including abbreviations. It is also important to use alternate spellings and word endings. This can be done using a number of strategies within the database:
Firstly, truncation involves putting an asterisk (*) in a word with a variable ending e.g. toxic* will search for toxic, toxicity and toxicology.
Secondly, using a question mark (?) in place of a single letter for words that have alternate spellings (such as American vs British English) will search for both spellings, e.g. p?ediatric will search for both paediatric and pediatric. The question mark replaces a single character or no character, allowing you to pick up alternate spellings. It should be noted that these terms can vary between databases, and so should always be checked prior to conducting a search.
You should also search for hyphenated terms with and without the hyphen as different authors may title their work differently.
An example to illustrate a search in Ovid Medline:
Let’s imagine we want to conduct a systematic review to assess the effect of primary percutaneous coronary intervention (PCI) versus streptokinase on mortality in myocardial infarction (MI) patients. (Clicking on the screenshots below will enlarge them).
Patients: myocardial infarction, MI, myocardial necrosis, heart attack
Interventions: primary percutaneous coronary intervention, PCI, percutaneous coronary intervention, primary PCI, reperfusion, re-perfusion, angioplasty
Comparisons: streptokinase, medical reperfusion, medical re-perfusion
Outcomes: mortality, death, survival
As you can see, each line represents a synonymous term that is searched for, with the corresponding number of articles found in the results column. These terms can be combined using ‘OR’ to ensure the database captures all of the articles relating to these terms.
For the final screenshot, all of the groups of terms are combined with ‘AND’ to ensure the database only displays results that are related to the combination of terms (i.e. about PCI and streptokinase in MI patients). In this example, it yielded a total of 254 results.
Further refining or increasing your results
Medical Subject Headings (or MeSH terms) are terms predefined by the database using human indexers in concordance with thorough protocols. MeSH terms encompass ‘Headings’, ‘Subheadings’, ‘Supplementary Concept Records’ and ‘Publication Characteristics’. Further definitions of these categories and relevant examples can be found on the National Library of Medicine’s website (https://www.nlm.nih.gov/bsd/disted/meshtutorial/introduction/04.html). The same website also provides detailed descriptions of how the MeSH system is structured and how this can be searched to further refine or expand your literature search. Using Medline as an example, you must select ‘Map term to subject heading’ to search for MeSH terms.’
If your search yields no results then it is important to re-evaluate your PICO model and broaden the search terms. Conversely, if the results are very expansive this will take considerable effort to screen the relevant information. It may be worth narrowing the search terms in order to achieve a more focused search.
AFTER THE SEARCH
Any more studies to include?
Once the formal search has been completed, you must now remove the duplicates and screen the titles/abstracts of the remaining results. This is a large task in itself, and information on how this can be done is beyond the scope of this blog.
Once competed, a further exploration for relevant studies can begin. There are a number of strategies that should be carried out at this stage.
- Citation tracking – This involves searching for studies that have referenced the included studies in their work, and as such, may be relevant to your systematic review. Most databases will have an integrated tool to perform this.
- Manual reference searching – Once you have finalised a list of applicable studies, you should go through their references individually and search for additional relevant studies.
- Contacting authors – Finally, authors of the included studies can be contacted to search for further results that may be pertinent to your review, or to ensure your search strategy has not missed any of their other work which may be useful.
At this point it is crucial that you finalise your list of included studies and document your search well so that you can refer to this in the future and during the rest of the review process.
Your SYSTEMATIC literature search is now complete!
Congratulations! If you have followed the various steps in this article, you are well on your way to completing a comprehensive systematic review! If you need further help, consult your librarian.
- Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011. Available from cochrane-handbook.org.
- Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses:The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097
No Comments on Conducting a systematic literature search
As I understand your text you suggest using all four categories of PICO in your search. I would argue that doing that is highly unusual in systematic reviews, in fact I can’t remember ever seeing it. I myself rarely use more than population and intervention in my searches.13th January 2018 at 9:56 pm
Apologies Saul, I made a typo in spelling your name!15th December 2017 at 12:42 pm
Great post. I would just emphasize that (as Paul says) it is a ‘strategy’ not a set approach. Web sites and web searches may, for example, be crucial sources for some review questions. See: G. Brunton et al ‘Finding relevant studies’ in Gough et al (2017) ‘Introduction to Systematic Reviews’ 2nd Edn. Sage Publications.15th December 2017 at 10:17 am
I would advise using both thesaurus terms and natural language terms simultaneously, rather than, as you seem to advocate, only including thesaurus terms to refine results.24th July 2017 at 11:44 am
It isn’t quite correct to say that controlled vocabulary terms are ‘created by the database itself’. In the case of MeSH, these are assigned by human indexers, following a well-described procedure.
It is perhaps also worth pointing out that different databases use different thesauri. One criterion by which search strategies can be judged is how well the searcher has translated terms between databases.
And, if you use only e.g. MeSH term you will miss new studies that haven’t been indexed yet.13th January 2018 at 9:52 pm
Your blogs are excellent!20th July 2017 at 12:15 am
Hi Saul, Great introduction to literature searching. Readers may also be interested in “Grey Matters: a practical tool for searching health-related grey literature,” a comprehensive tool developed by health librarians at CADTH. Grey Matters and it’s companion user guide are available free of charge at http://www.cadth.ca/resources/finding-evidence/grey-matters.18th July 2017 at 3:41 pm