Research: increasing value, reducing waste
Posted on 27th January 2014 by Alice Buchan
In theory, medical research is simple. From finding a mechanism, drugs are developed first in animal models, then put through successive stages of human trials before licensing; these trials are brought together into systematic reviews, and evidence-based practice using these mean that every patient gets the treatment that the evidence suggests is best. Sadly, it’s not always that simple in the real world. I’ve previously written about situations where there simply isn’t any evidence to base that practice on (here), but even within this paradigm, all may not be as rosy as it seems.
A huge amount of biomedical research is conducted globally each year, funded by charities, government bodies, universities and industry. Whilst funding remains extremely competitive, the overall amount spent is huge. In the UK, the Medical Research Council (MRC) spent £766.9 million on research in 2012-13 [1], and across the Atlantic in the US, the National Institutes of Health (NIH) spent $30.9 billion on research [2]. There is understandable concern that some of this money is not being used as efficiently as it perhaps could be, and that waste should be kept to a minimum.
This series of papers in The Lancet looked at just that, and through the five articles, they tackle a range of research areas, including:
- setting research priorities [3]
- research design, conduct, and analysis [4]
- biomedical research regulation and management [5]
- inaccessible research [6]
- incomplete or unusable research [7]
Setting research priorities.
In terms of efficiency and value for the funding that goes into healthcare research, priorities are key, as arguably the funders get poor value if the money is being spent on the wrong thing. That said, priority setting is not always easy, as everyone with an interest in a condition, including both patients and clinicians, will argue that their condition merits more funding and more research. One way of tackling this has been through the James Lind Alliance, which brings together stakeholders to help set priorities together [8]. In The Lancet article, the authors make the following 4 recommendations:
- “First, ways to improve the yield from basic research should be investigated.
- Second, the transparency of processes by which funders prioritise important uncertainties should be increased, making clear how they take account of the needs of potential users of research.
- Third, investment in additional research should always be preceded by systematic assessment of existing evidence.
- Fourth, sources of information about research that is in progress should be strengthened and developed and used by researchers. Research funders have primary responsibility for reductions in waste resulting from decisions about what research to do.” [3]
Research design, conduct, and analysis
This may seem self-evident, but one way to ensure the best use of limited resources is to make sure that research that is funded is done as well as possible. The authors of this paper found that this was lacking particularly in terms of analysis and statistics [4]. One major issue identified is that researchers, and doctors in particular, often do not get a thorough training in statistics and analysis, leading to scary statistics such as the following:
“These issues are often related to misuse of statistical methods, which is accentuated by inadequate training in methods. For example, a study of reports published in 2001 showed that p values did not correspond to the given test statistics in 38% of articles published in Nature and 25% in the British Medical Journal. ” [4]
They identified 4 key problems with research design, conduct, and analysis: poor protocols and designs, poor utility of information, statistical power and outcome misconception, and insufficient consideration of other evidence [4].
Biomedical research regulation and management
Biomedical or pre-clinical research is often what clinical studies are based on. However, much of what looks promising at this stage does not translate into clinical benefit. Al-Shahi Salman et al. have proposed four recommendations to improve value and reduce waste at this stage:
1. “People regulating research should use their influence to reduce other causes of waste and inefficiency in research” [5]
2. “Regulators and policy makers should work with researchers, patients, and health professionals to streamline and harmonise the laws, regulations, guidelines, and processes that govern whether and how research can be done, and ensure that they are proportionate to the plausible risks associated with the research” [5]
3. “Researchers and research managers should increase the efficiency of recruitment, retention, data monitoring, and data sharing in research through the use of research designs known to reduce inefficiencies, and do additional research to learn how efficiency can be increased” [5]
4. “Everyone, particularly individuals responsible for health-care systems, can help to improve the efficiency of clinical research by promoting integration of research in everyday clinical practice” [5]
Inaccessible research
As with many of the problems identified in this series of articles, this one is almost self-evident. Problems with inaccessible research have attracted a lot of attention recently, with The BMJ Open Data campaign sparked by Cochrane reviewers being unable to access unpublished trials of Tamiflu [9]. The All Trials campaign also seeks publication of all trials and reporting of all results [10].
Publication bias is a known issue, and the authors of this Lancet paper also note a submission bias: “When reported, clinical trials with positive results appear in journals about 1 year earlier than do those with results that are not positive.12 Reporting of trials that show no significant effect can be delayed for several years (table), even when the findings have substantial global implications. Although widely suspected, no empirical evidence is available that journals preferentially publish reports showing positive results rather than those with non-positive results (figure 1),3 indicating that investigators do not submit reports of studies with negative results.” [6]
They have 3 recommendations to help tackle the issue of inaccessible research:
- “First, institutions and funders should adopt performance metrics that recognise full dissemination of research.” [6]
- “Investigators, funders, sponsors, regulators, research ethics committees, and journals should systematically develop and adopt standards for the content of key study documents and for data-sharing practices” [6]
- “Funders, sponsors, regulators, research ethics committees, journals, and legislators should endorse and enforce study registration, wide availability of full study information, and sharing of participant-level data for all health research.” [6]
Incomplete or unusable research
Inefficiency is the focus of this piece [7]. Incomplete or unusable research is clearly wasteful, and their focus is on good reporting: “ Bradford Hill2 suggested that reports of research should answer four questions: what questions were addressed and why, what was done (the materials and methods), what was shown (direction, size, and uncertainty of effects), and what the findings mean (in the context of other research). Answers should be readable, complete, and make allowances for different audiences. However, most research reporting falls far short of these ideals” [7] The recommendations they make also focus predominantly on improving reporting, making it easier for others to use the research in question.
Links
[1] Medical Research Council. Facts and Figures. http://www.mrc.ac.uk [Accessed online 24 January 2014]
[2] National Institutes of Health. NIH Budget – About NIH. http://www.nih.gov/about/budget.htm [Accessed online 24 January 2014]
[3] Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, Howells DW, Ioannidis JP, Oliver S. How to increase value and reduce waste when research priorities are set. Lancet 2014;383:156-65
[4] Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, Schulz KF, Tibshirani R. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014;383:166-75
[5] Al-Shahi Salman R, Beller E, Kagan J, Hemminki E, Phillips RS, Savulescu J, Macleod M, Wisely J, Chalmers I. Increasing value and reducing waste in biomedical research regulation and management. Lancet 2014;383:176-85
[6] Chan AW, Song F, Vickers A, Jefferson T, Dickersin K, Gøtzsche PC, Krumholz HM, Ghersi D, van der Worp HB. Increasing value and reducing waste: addressing inaccessible research. Lancet 2014;383:257-66
[7] Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014;383:267-76
[8] James Lind Alliance. How the JLA works. http://www.lindalliance.org/Introduction.asp [accessed online 26 Jan 2014]
[9] BMJ. Tamiflu campaign. http://www.bmj.com/tamiflu [accessed online 26 Jan 2014]
[10] All Trials. All Trials. http://www.alltrials.net [accessed online 26 Jan 2014]
No Comments on Research: increasing value, reducing waste
Interesting post, I learned something new today. Thanks for sharing your knowledge
8th May 2023 at 7:38 amThank you so much for sharing such useful information over here with us. This is really a great blog have you written. I really enjoyed reading your article. I will be looking forward to reading your next post.
2nd May 2023 at 10:05 amClinical research enables researchers and doctors to find new and better ways to understand, detect, control and treat human diseases. A clinical research study is a way to find answers to difficult scientific or health questions. As I am student of clinical research. I found your article very helpful for my project research. Thanks for putting out top notch content all the time. I would like to be here again to find another masterpiece article.
23rd March 2019 at 3:53 amThanks for drawing attention to the Lancet series on waste, Alice.
27th January 2014 at 11:25 amA brief comment on the beginning of your blog: although some treatments originate by identifying a possible mechanism of action, which is then tested in animals, a very substantial proportion of useful treatments – drug and non-drug – don’t fit this model. successive stages of human trials before licensing
Hi Iain,
I really enjoyed reading the series, and liked that it had a bench-to-beside and beyond perspective.
Thanks for reading the blog and for your helpful comments. I tried to mention pre-clinical research looking for mechanisms as one example, and I was keen to use this since the paper on biomedical research highlighted the huge number of treatments that look promising pre-clinically but are not successful, and the identification of a mechanism or drug target preclinically could be considered one paradigm.
I agree that it is important to consider the very large number of drugs that are not developed in this way as well, and looking at differences in success between human trial stages as well as the jump from animal to human, and whether that could be due to inefficiency and waste.
Thanks,
Alice
27th January 2014 at 4:42 pm