The bias of language
Posted on 26th June 2015 by Katherine Stagg
What is language bias?
The practice of Evidence-Based Medicine can only be effective if the evidence that decisions are based on is accurate. The accuracy of the data can be affected by an almost infinite list of things but one group of problems are the reporting biases. Reporting bias occurs when the publication of some results is favoured over others, most commonly positive results being published more often than negative results. A subtype of reporting bias is that of language bias, which the Cochrane Bias Methods group defines as ‘the publication of research findings in a particular language, depending on the nature and direction of the results’1.
How language bias creates problems
It has been speculated that results produced in non-English speaking countries are more likely to be published in English-language journals if they are positive, but other language journals if they are negative. Since the majority of large, international journals are published in English there is potentially more incentive to publish exciting positive results in this language. This creates a biased level of exposure to either positive or negative results to those reading articles in only English or another language.
Beyond individuals papers though, language bias also has the potential to create flawed systematic reviews. This occurs when systematic reviews restrict their search for relevant papers to those published in English only. This creates a situation in which only a subset of papers is included, creating an incomplete data set. Given the previously mentioned issues regarding language bias in publishing positive and negative results, there is a serious risk that the subset of papers is not only incomplete but is also biased one way or the other.
Two studies have looked into the impact that language bias has on the results of systematic reviews2,3. Both found that the inclusion or non-inclusion of studies not published in English did not have a significant effect on the outcome of systematic reviews. However, one found that there was a statistically significant difference in the proportion of positive and negative result papers being published in English versus other languages, so although the overall result is relieving caution still needs to be exercised. It was also found that systematic reviews that included papers in all language were more likely to be of a higher quality.
Overcoming the bias
Language bias can be overcome in one of two way; ensuring that reviews are not restricted solely to English-language papers or by consistently publishing papers in only one language. Both of these are slowly happening. A 2010 paper published in the BMJ looking at biases in systematic reviews found that only 2 out of 283 Cochrane reviews were language-restricted4. There is also an increasing trend to publish in English, with publication in other languages such as German gradually decreasing over time1. The problem of language bias may therefore be becoming less significant, but is still one to be aware of and avoid.
References
- Addressing Reporting Biases, Cochrane Bias Methods Group, Feb 2013
- Direction and impact of language bias in meta-analyses of controlled trials: empirical study., Jüni P1, Holenstein F, Sterne J, Bartlett C, Egger M., Int J Epidemiol., 2002 Feb;31(1):115-23
- The inclusion of reports of randomised trials published in languages other than English in systematic reviews., Moher D1, Pham B, Lawson ML, Klassen TP., Health Technol Assess., 2003;7(41):1-90.
- The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews., Kirkham JJ1, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, Williamson PR., BMJ., 2010 Feb 15;340:c365.
No Comments on The bias of language