GRADE-CERQual: Confidence in the Evidence from Reviews of Qualitative Research
Posted on 4th September 2019 by Dolly Sud
Qualitative data as an evidence base
We would all agree that the best available evidence should be used as the basis for decisions about health, social care, and other policies, programmes and interventions [1]. Although different types of evidence will be viewed as having varied importance by stakeholders it makes sense that decisions should be informed by a broad range of evidence.
Implementation of guidelines, recommendations, policies and programmes (which often involve complex interventions) can have impact across systems within healthcare, education and social care.
Effective implementation may be impacted upon by many factors including:
– importance individuals attach to various outcomes
– acceptability or feasibility of the intervention
– the cost-effectiveness and investment of resource required
– scalability and related considerations
The reasons for lack of or poor quality implementation of evidence-based guidelines or policies may also require a diverse range of evidence, particularly in situations where constraints exist for resources (e.g. low-income countries). Qualitative research data provides fundamental information to meet this need.
The aim of qualitative research is to explore views, perspectives, opinions and experiences of individuals of the world around them including health and illness and health and social care.
Qualitative evidence syntheses/systematic reviews of qualitative research are increasingly being undertaken and established and well-developed methods for undertaking such reviews now exist [2,3,4]. Evidence from such reviews is more widely being used to inform decisions, for example, as part of health technology assessments to augment evidence on intervention effect and use of resources.
The use of qualitative evidence is far and wide reaching including:
– Evidence to decision frameworks such as DECIDE [5]
– Evidence based policy briefs such as SURE [6]
– Decisions on implementation strategies to use.
The broader application and use of qualitative evidence by bodies such as the National Institute for Health and Care Excellence (NICE) in the UK, the World Health Organization (WHO) has brought attention to the need to have approaches to assist users to make decisions about how much emphasis can be given to such evidence in the decision making process [7].
What methods exist for assessing the confidence from reviews of qualitative research?
Perhaps surprisingly despite the dearth of qualitative methods and data that exist, prior to the development of GRADE-CERQual no accepted, structured method existed for assessing confidence in the evidence from qualitative evidence syntheses! The lack of such methods may constrain the use of qualitative evidence to inform decision-making as we are not able to use this evidence with confidence.
What do we mean by confidence in evidence?
Essentially, this is an assessment of the extent to which the findings of a review are a reasonable representation of the phenomenon of interest. This will include the extent to which the findings from a review are likely to be considerably different to the phenomenon of interest.
For example, if the findings from a review suggest that a new healthcare intervention is highly acceptable to mental health service users then we will be highly confident in these findings (in other words findings are a reasonable representation of the acceptability to mental health service users). Assuming that the benefits outweigh the risk then decision-makers may make a recommendation to implement the intervention based on this. Conversely, if there is low confidence in the findings then it will not be clear that the intervention is acceptable and decision-makers may not make the recommendation to implement.
GRADE-CERQual [8] involves an assessment of each individual review finding in terms of four components:
1. methodological limitations
2. coherence
3. adequacy of data
4. relevance
The assessments of the four components collectively contribute to an overall assessment of whether findings from a qualitative evidence synthesis provide a reasonable representation of the health or social care issue, intervention, or programme (phenomenon) of interest.
This approach is comparable to that for GRADE [9] for effectiveness where review authors assess the confidence, or certainty, in the estimates of effect for each critical and important outcome by evaluating risk of bias, directness, inconsistency, imprecision, and publication bias
Where can I find the guidance?
- GRADE CERQual main guidance
- The GRADE working group
- BMC implementation science has an excellent seven-part series providing guidance on how to apply the CERQual approach
Conclusions
Qualitative evidence is increasingly being used within decision making including in the development of guidelines and health technology assessments; it performs the role of representing the views and experiences of a wide range of stakeholders.
GRADE-CERQual allows an assessment of how much confidence to place in findings from qualitative evidence syntheses. It complements other ‘Grading of Recommendations Assessment, Development, and Evaluation’ (GRADE) tools for assessing how much confidence to place in evidence on the effectiveness and harms of interventions and their resource use and in evidence about diagnostic tests. You can read more about GRADE in this separate blog on the website: GRADE and quality of evidence.
Abbreviations:
CERQual: Confidence in the Evidence from Reviews of Qualitative research
DECIDE: Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (an EU-funded research project)
GRADE: Grading of Recommendations Assessment, Development, and Evaluation
