Reviewing the evidence: what method should I use?

Posted on 8th October 2019 by Dolly Sud

Overview

Within my field of practice (mental health/psychiatric pharmacy) I identified a priority issue relating to healthcare. Approximately 2 in 3 deaths in those who have severe mental illness (SMI) (e.g. schizophrenia) are due to physical illnesses such as cardiovascular disease and can be prevented. Excess premature mortality rates (up to 20 years) in those with SMI are more than three times higher in England compared to the general population, with recent analyses suggesting that this gap is widening. I wanted to know what role or impact pharmacy could have in addressing this inequality; this has formed the basis of my research/PhD study.

The first stage was to review the current literature on this topic. An easy sentence to write but a very involved process to execute! In addition, other than undertaking a few research modules earlier on in my studies, my experience of research was fleeting if not virtually non-existent. I understood that a well-conducted, rigorous literature review should seek to use all the relevant evidence and to provide a full picture but needed also to be systematic and transparent. But what does that mean? What does it mean to be systematic and transparent? And are there good reference sources to start with that would at least help guide me initially? If someone wanted to use the literature review as the basis for making decisions then how do I ensure that I have minimised bias in selecting the evidence for review? Some of you may relate to this predicament!

Mantra: minimize bias, be systematic, be transparent, be reliable

Bear in mind that when a review is conducted the onus is on the reviewer(s) to present findings which minimize bias; in other words minimize inaccurate judgement about what the evidence shows. For example, only selecting studies that support the intervention being considered and excluding those studies that show the intervention less favourably is biased. The review should be conducted such that each stage is systematic. The reader of the review should be able to clearly understand the steps that have been taken. A well written protocol can meet this need (be transparent). In addition if someone is to repeat the process using the method they should find the same results (also known as reliability). Of course if the process is documented as completely as possible this reduces the likelihood of questions being asked later and increases the confidence of readers in results and recommendations.

Minimize bias, be systematic, be transparent, be reliable.

Initial searches on the internet provided an overwhelming amount of information which at the time felt to me more suited to seasoned researchers. At this point I took a very deep breath and sought guidance from my supervisor. I was directed to some reference sources which provided guidance that facilitated subsequent decisions about how I could undertake the review to meet the needs of the research [See the Reference list at the bottom of the blog]. Whilst this list is not exhaustive I feel it is a great place to start.

Booth [1] states that in choosing or deciding which method might be appropriate for review of published evidence the following must be considered:

1. The type of review question you are asking
2. The type and quantity of studies available to answer your question
3. How your final review is going to be used
4. The skills, resource and expertise of your team

In addition, when undertaking any synthesis of evidence there is a reality of making a trade-off between rigour, timeliness and feasibility. I would recommend the sources I have referenced below. The first reference source I found particularly useful as it provided a concise overview and included a table of each type of method, what the method is, when it should be used, how it is done and approximate time taken.

I wish all of you the best of luck with your evidence reviews. When you first start undertaking evidence reviews, having a few ‘go to’ reference sources that you can rely on is like a recipe book that you keep going back to time and time again.

References (pdf)

You may also find the following blogs interesting:

Conducting a systematic literature search

CASP Tools for Randomised Controlled Trials (RCT)

GRADE and quality of evidence

Tags: