The perverse incentives in academia to produce positive results

Posted on 6th November 2023 by Rosario De Feo

Academic publishing is without doubt one of the pillars of the scientific community, enabling the spread of knowledge and advancement of human understanding. Scientific literature, especially those dealing with topics such as medicine and health, is a major field of science: all the drugs, therapies, and remedies we use are based on it. Evidence-based medicine (EBM) itself is based on it! Therefore, we can say without any exaggeration that our lives, in one way or another, are based on it.

Regrettably, as with all things human, a form of corruption is ever more present in academic publishing too, which is something we should be worried about. The concept of research being one of the most reliable activities in the scientific field has been undermined by a growing phenomenon: the perverse incentives in academia to produce positive results.

The erosion of academic integrity

Unfortunately, the practice of perverse incentives is becoming widespread in the academic environment. If a critical mass of scientists become untrustworthy, a tipping point is possible in which the scientific enterprise itself becomes inherently corrupt and public trust is lost, risking a new dark age with devastating consequences to humanity.  The true incidence is difficult to predict due to the number of cases of undetected misconduct being estimated each year.

For example, the news that Dr Elizabeth Bik (Scientific Integrity Consultant) criticized the President of Stanford University and the Italian Health Minister due to their involvement in publishing scientific papers with duplicated or recycled photos caused quite a stir. Another example of the danger of predatory journals is represented by the story of a patient with cancer. This woman had exhausted all other forms of treatment when her alternative medicine practitioner suggested an article that seemed to be reliable. This paper of course represented to her a final option for treatment, but her son-in-law realized that it came from a predatory journal, and its validity was unlikely to have been vetted.

Perverse incentives and the threat to trust

Clearly, perverse incentives can encourage unethical behaviour to manipulate quantitative metrics: an uncontrolled perverse incentive system can create a climate in which participants feel they must ‘cheat to compete’.

The only people who can survive in this environment are those passionate about what they’re doing and have the self-confidence and competitiveness to just go back again and again, persistently applying for funding (Robert Waterland, Baylor College of Medicine) (Harris and Benincasa, 2014).

The definition of ‘perverse incentives’ is meant in different ways, but this phenomenon is also spreading throughout the scientific community because of the very rules by which researchers are judged, including career progression. The goal of measuring scientific productivity has given rise to quantitative performance metrics including publication count, citations, combined citation-publication counts (e.g., h-index), journal impact factors (JIF), total research dollars, and total patents. These quantitative metrics now dominate decision-making in faculty hiring, promotion and tenure, awards, and funding (Abbott et al., 2010; Carpenter et al., 2014). Because these measures are subject to manipulation, they are doomed to become misleading and even counterproductive, according to Goodhart’s Law, which states that ‘‘when a measure becomes a target, it ceases to be a good measure’’ (Elton, 2004; Fischer et al., 2012; Werner, 2015).

A comprehensive meta-analysis of research misconduct surveys between 1987 and 2008 indicated that 1 in 50 scientists admitted to committing misconduct (fabrication, falsification, and/or modifying data) at least once, and 14% knew of colleagues who had done so (Fanelli, 2009). These numbers are likely an underestimate considering the sensitivity of the questions asked, low response rates, and the Muhammad Ali effect (a self-serving bias where people perceive themselves as more honest than their peers) (Allison et al., 1989). In life science and biomedical research, the percentage of scientific articles retracted has increased 10-fold since 1975.

One of the most common examples of this corrupted system is the case in which a particular paper on a specific drug or supplement is required by the factory producing the drug itself – the paper is designed expressly to bring scientific evidence to that drug/ supplement. When scientists manipulate data, they have to obtain the results they want, even if the experiment went wrong.

All this has led to the increasing number of predatory journals – journals which publish almost all submitted manuscripts because their main aim is not that of publishing robust, reliable, good-quality evidence but earning money. These kinds of journals are often ‘open access’, meaning it is a journal everyone can see and study without paying a fee. On the contrary, traditional publishing models generally involve expensive subscription fees and access barriers, limiting the dissemination of research to a privileged few. Open access publishing, which aims to address this issue, has introduced new challenges in the form of author fees, where researchers are often required to pay substantial charges to publish their work.

Furthermore, this system often supports popular (‘me too’) projects and not the innovative ones, because new ideas would be less cited in the present moment. This is also important because the journals themselves need to be cited. Journals undergo the same ‘treatment’ as researchers with regards to gaining prestige: the more their works are cited, the more prestige (measured in terms of impact factor for example) they have.  In this way, the paper is more widespread, and the possibility of being cited grows. However, if open-access journals do not receive money from subscriptions, they still have to earn: here is why the author themselves have to pay for having their work published. It’s useless to say that it’s a vicious circle because the tight rules of academia want the researcher to publish more, so they fight each other to publish the most they can. This can lead to the quality of paper being inevitably low because new work is always asked of. This perpetuates a cycle where scholars, under pressure to publish, fall prey to the predatory journal industry, further compromising the quality and integrity of academic research.

Quantity is therefore also preferred to quality because open-access journals mask themselves as legitimate outlets using misleading names, fake editorial boards, and deceptive promises of rapid publication to lure unsuspecting scholars, exploiting researchers’ desperation to publish quickly.  An overemphasis on quantity is problematic because accepting less scientific rigour in statistics, replication, and quality controls, or a less rigorous review process, would produce a very high number of articles, but after considering costly setbacks associated with a high error rate, true progress would also be low.

Science is a human endeavour, and despite its obvious contribution to the advancement of civilization, there is growing evidence that today’s research publications too frequently suffer from a lack of replicability, rely on biased data sets, apply low or substandard statistical methods, fail to guard against researcher biases, and their findings are over-hyped. It appears clear how every day they are compromising the credibility of academic research, even the good and well-conducted ones. If a subpar or even pseudoscientific work is published without problem, it not only leads to misinformation given to the public but erodes trust in scientists and doctors as a whole. In addition, if one of these low-quality studies becomes very cited, it may also lead to a wrong assessment of the treatment of a disease.

What can we do to avoid these issues?

Radical changes are required to improve the system’s efficiency, liberate creativity, and encourage innovation from below. Peer review needs far greater public openness and feedback from the broader research community. It is clear that this creation is a threat to the future of science and, unless immediate action is taken, we run the risk of ‘‘normalization of corruption’’ (Ashforth and Anand, 2003), creating a corrupt professional culture.

According to several scientists, hypothetical optimum productivity lies somewhere between quality and quantity, and our current practices (enforced by peer review) may evolve to be near the optimum in an environment with fewer perverse incentives.

Reforming how researchers are judged and reducing the spread of predatory journals is necessary but, without doubt, very difficult. One possible solution to combat poor scientific practice may be the introduction of a quantitative metric that measures a scientist’s commitment to transparency and open access (the author suggests this is named the ‘Altman index,’ in remembrance of Professor Doug Altman and his commitment to scientific integrity) (Richards et al., 2021). Another initiative that is trying to limit the ‘power’ these journals have is making a blacklist with all known predatory ones, to help young and willing-to-publish scholars/researchers avoid their ‘trap’. According to Grudniewicz, A. et al (2019) these are some steps researchers can take to avoid predatory journals:

1. Understand the Issue: Researchers should be aware of the existence and prevalence of predatory journals that prioritize self-interest over quality scholarship
2. Use Clear Definitions: The article proposes a consensus definition for predatory journals, helping researchers recognize their characteristics, such as false information, lack of transparency, and aggressive solicitation
3. Check Journal Quality: Researchers should thoroughly assess the journal they plan to submit their work to. Look for signs of false information, deviations from best practices, lack of transparency, and aggressive solicitation
4. Use Trusted Resources: Rely on trusted sources and guidelines like “Think. Check. Submit.” to help identify reputable journals
5. Beware of Aggressive Solicitation: Be cautious of journals that excessively solicit submissions through flattering emails or out-of-scope topics
6. Verify Claims: Check the journal’s claims of indexing, impact factors, and editorial board legitimacy. Verify the journal’s history and practices
7. Participate in Educational Campaigns: Engage in educational campaigns and initiatives to raise awareness of predatory publishing in the research community

By understanding the issue, using clear definitions, and exercising caution and due diligence, researchers can minimize the risk of falling victim to predatory journals and contribute to the effort of combating this problem in the academic and scientific publishing world.

Conclusions

In conclusion, we can only hope for all the researchers and scholars involved in research what Nobel laureate Richard Feynman stated:

… the good luck to be somewhere where you are free to maintain the kind of integrity and where you do not feel forced by a need to maintain your position in the organization, or financial support, or so on, to lose your integrity.

References (pdf)

Tags: