Publication bias: The answer to your research question may be sitting in somebody’s file drawer

Posted on 7th August 2018 by Andrés Viteri-García

Publication bias, also known as dissemination bias, is defined as:

the publication or non-publication of research findings based on the nature and/or direction of their results (1).

Studies reporting results that are perceived as positive, statistically significant, or favouring the study hypothesis are more likely to be published. Several projects assessing the rate of publication of randomised controlled trials (RCTs) protocols registered prospectively in ClinicalTrials.gov, submitted to ethics committees, or of abstracts presented in conferences or congresses, show that publication rates range between 60 to 98% for studies with positive results and between 20 to 85% for studies with negative results (2).

Publication bias is just one of the many shapes of an overarching phenomenon: reporting bias, which also encompasses time lag bias, location bias, language bias, multiple publication bias, citation bias, and selective outcome reporting, among others (Table 1) (1,3). Reporting bias is generated from the perception that positive, statistically significant results will have a greater impact on the scientific community and in clinical practice, with the corresponding effect on researchers’ academic careers.

Table 1: Types and definitions of reporting bias (from the Cochrane handbook)

To further explore the issue of publication bias, we have contacted Dr. Hector Pardo-Hernandez (@hpardoh), researcher at the Iberoamerican Cochrane Centre, who recently defended his thesis project ‘Strategies to prevent and control dissemination bias of controlled clinical trials’.

• Why is publication bias a problem?

When a study is not published, its visibility and potential impact is diminished, disavowing the contribution made by patients who oftentimes participate without the certainty that they will receive a direct benefit. In addition, publication bias may result in redundant research and in a consequent waste of economic resources. As such, a 2012 study on reporting completeness and time to publication of RCT protocols in ClinicalTrials.gov found that less than half had published results; for those who did have published results, important data such as adverse events and the flow of participants was poorly described (4). Moreover, the efficacy of the research process is limited by a lack of access to essential data needed for, just to cite an example, the development or updating of systematic reviews (SRs).

• How does publication bias affect the practice of evidence-based medicine?

Publication bias generally leads to the dissemination of positive results that support the study hypothesis. As such, it translates into an overestimation of treatment effects during SR development (type 1 error). Publication bias therefore undermines the validity of SRs and reduces confidence in evidence obtained from RCTs.

• Speaking of systematic reviews, what are the strategies that authors can implement to prevent or control publication bias?

The Methodological Expectations of Cochrane Interventions Reviews (MECIR) standards recommend extensive searching during systematic review development in several databases, including but not limited to Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Cochrane Specialized Registers, and databases relevant to the research question, such as CINAHL for nursing or PsycINFO for interventions in psychology. In addition, authors are advised to consult ClinicalTrials.gov and other prospective trials registers and registries, as well as grey literature and the reference lists of eligible studies.

• What about journals not found or indexed in the main databases?

There are different initiatives around the globe to tackle this shortcoming. For instance, at the Iberoamerican Cochrane Centre we lead an initiative through which references to randomized controlled trials (RCTs) are identified by checking Spanish and Latin American journal articles by hand, page by page (handsearching). Retrieved references are then submitted to CENTRAL for their potential inclusion in SRs.

• Does the problem affect just RCTs?

No, publication bias comes in many forms and shapes. Different studies have identified publication bias among SRs and qualitative research, among others (5,6,7). For instance, a 2006-2007 survey among systematic review corresponding authors found that, collectively, the 348 respondents had a total of 199 unpublished systematic reviews (with a median of 2.0 unpublished systematic reviews per author) (8).

• Gordon Guyatt and Ben Djulbegovic have cited publication bias as one of the main threats to the practice of evidence-based medicine (9). What initiatives are being implemented to tackle publication bias?

Initiatives such as ‘AllTrials’, which calls for prospective registration of clinical trials and for full reporting of methods and results (10), ‘REduce research Waste And Reward Diligence (REWARD)’ which sets research priorities and looks for the complete report of any investigation (11), and the ethical standards provided by the Code of Conduct of the Committee on Publication Ethics (12) and World Association of Medical Editors (13), have advocated for the publication of all RCTs results regardless of their direction or magnitude (9,14). In addition, one of the proposals that currently generates more interest is the deposit of anonymous individual patient data from RCTs in open access registers. Anonymous individual patient data facilitates the reproducibility of analyses and makes us less dependent on full-text articles to obtain RCT results.

Other strategies, such as open access publishing, can reduce publication bias by enhancing access to full-text articles and facilitating the dissemination of negative, not statistically significant study results. On the other hand, open access publishing may exacerbate the problem in developing countries, by imposing an economic burden for publishing that may not be assumed by unfunded researchers (15).

References

(Featured image: Photo by Sanwal Deen on Unsplash)

Tags: