The imperfections of Evidence-Based Medicine

Posted on 21st May 2015 by Katherine Stagg

While most, if not all, members and readers of Students 4 Best Evidence will be keen advocates of the practise of Evidence-Based Medicine (EBM) it would be foolish to suggest that it is without flaws. Understanding these weaknesses is important in order to improve our practice of EBM, both at an individual and wider level.

What is it?

‘Time to abandon evidence based medicine?’ is an hour long Youtube video of a lecture by Prof David Nealy from the University of Cardiff. It is a ‘challenge to EBM’ and focusses on the errors that occur in the practice and implementation of EBM and the dangerous consequences that they can have. Key issues that Prof Nealy focusses on are the disregard for the personal experience of doctors, the manipulation of statistics by pharmaceutical companies to make the use of drugs appear evidence-based when it is not and the lack of publication of negative results.

The paroxetine scandal

These problems are well explained with real-life examples. Particularly interesting is the example of the paroxetine scandal, in which Prof Nealy describes data from pharmaceutical companies that was manipulated and selectively published in order to give the appearance that paroxetine is a safe, effective drug for the treatment of depression in adolescents, hiding the fact that it increased suicide rates.

The paroxetine scandal shows that EBM can be manipulated for negative reasons, and is a good reminder that proper use of EBM requires careful evaluation and assessment of the evidence. Reading published research alone is not enough, we must all ensure that we really believe and trust that it is based on sound experimental design and statistical evaluation before we apply it to our clinical practice.

The problem of personal experience

Over time doctors see many hundreds of patients and will experience how they each react to certain drugs and treatments. This leads to the development of personal preferences as to how patients should be treated. EBM combats the idea that treatment should be based solely on personal experience. This is a good thing, because it means that clinical decisions are based on actual trial evidence and prevents personal biases influencing treatment. The downside that Prof Nealy discusses is that doctors have little way for their knowledge about the side effects that patients experience to be taken seriously. While there are reporting mechanisms, reports by doctors can be dismissed as anecdotal. This means that information about drugs and treatments can be lost or ignored. To try and combat this Prof Nealy has established his own independent drug safety website in which side effects can be reported and collated.

Overall verdict

This is a good video that covers the weaknesses of EBM well, with stimulating and relevant examples used to illustrate points. I would recommend it to medical students who are interested in EBM or research. It has some really valuable information (for example, I didn’t know that FDA reports are based on published and unpublished data, whereas Cochrane reviews have access only to published data) and is well argued. While I’m not going to end my support for EBM any time soon it certainly raises some good points and it’s useful to know what the opponents of EBM believe. It’s also important as a reminder that safe, effective practice of EBM requires continual assessment of the evidence and that published results should not always be taken at face value.

Links

Interesting articles about problems with EBM:

The Role of Experience in Science-Based Medicine, H Hall, Science-Based Medicine, 2011

https://www.sciencebasedmedicine.org/the-role-of-experience-in-science-based-medicine/

The Limitations of Evidence-Based Medicine—Applying Population-Based Recommendations to Individual Patients, JJ Goldman, AMA Journal of Ethics, 2011.

http://journalofethics.ama-assn.org/2011/01/jdsc1-1101.html

Evidence Based Medicine: a movement in crisis?, BMJ, 2014.

http://www.bmj.com/content/348/bmj.g3725

The Paroxetine Scandal:

http://www.drugwatch.com/paxil/

http://www.nursingtimes.net/whats-new-in-nursing/special-report-the-seroxat-scandal/892270.article

Tags: