The scientific world does not need more bad quality papers

Retractions are science’s way of correcting the scientific record by recalling works which have been shown to be problematic. The reasons for retractions range from plagiarism and fake data to honest errors in analysis.

In recent times, the number of articles being retracted has exploded, reaching a record 10,000 papers in 2023, according to a study by Nature. This suggests there has been an increase in substandard work being published.

This rise in retractions corresponds to an increase in the pressure for academics to publish. Fuelled by rankings and research metrics, researchers are publishing at a prolific rate.

The rate of increase in the number of articles being published is unsustainable and requires an urgent change in the ways we evaluate science.

In a recently published preprint, we found strong evidence that countries that have increased their publishing the most in recent times have experienced a higher proportion of retraction notices in the last three years. This trend is very worrying as it is evidence that commonly used research metrics may be driving researchers to publish a greater quantity of papers, leading to a lot of shoddy science.

Retractions by country

While retractions happen for a multitude of reasons, most are due to some form of research misconduct (although not necessarily by the author) or errors found in the articles. Looking at retraction statistics can thus provide a glimpse of the state of research misconduct in the world.

The Retraction Watch Database, which has been made public by Crossref, offers the biggest repository of retractions. We decided to look at the retraction notices that countries received in the last three years (2022-24). To be able to compare different countries, we looked at retraction notices per 1,000 publications as a measure. We call this the retraction notice rate.

In Figure 1 below we see the countries listed according to retraction notice rates. We found a correlation between countries with high retraction rates in our study and the 2023 Nature study by features editor Richard van Noorden.


Figure 1

However, there are a few countries which did not appear in the Nature article. They are Ethiopia, Bangladesh and Iraq. The reason for the discrepancy in the two lists is due to the different cut-off rates used by the two studies.

The Nature article chose only to include those countries that had published more than 100,000 articles in the period 2003-22 that it covered, and these countries didn’t reach that volume at the time. Van Noorden said: “These countries do appear high up when one analyses retraction rates – it’s just that their total article output was smaller than the other countries.”

Van Noorden says our study “usefully brings the analysis up to date for 2022-24, and in particular finds an intriguing correlation between countries that rapidly expanded publications over 2019-23, and countries that then had high rates of retraction notices”.

He notes that actual rate details can differ depending on methodological choices, such as which dataset one uses to count a country’s total publications, or whether one chooses to analyse rates of retraction notices (as in the new study) or retracted articles (as in the Nature work).

Nevertheless, similar countries appear high up in all cases. Further analysis might examine whether specific authors or institutions are particularly responsible for high retraction rates, he adds.

Our study shows countries from East Africa, the Middle East and the Indian subcontinent have the highest retraction notice rates, with Ethiopia, Saudi Arabia and Pakistan at the top.

However, this analysis comes with certain caveats:

• Some countries might figure high on the list as they may have an active community of science sleuths working to weed out bad papers. France, for example, is one of the leaders in the research integrity field and the work of their sleuths brought about a lot of retractions in 2024.

• There could be some bias at play as countries known to have higher retraction rates can come under greater scrutiny, leading to more retractions.

Nonetheless, this list is a good starting point for further investigation and interrogation.

We looked at how much countries have increased their publication numbers by in recent years. Remarkably, we see that the countries with the highest retraction notice rates are also the ones that have increased their publishing prolifically in the last five years.

If a country publishes more articles, it is natural that it will get more retraction notices. However, in Figure 2 below we see that the proportion of articles with retraction notices is greater, not just the absolute number of retractions. This is worrying as it suggests that pressure to publish more articles may be leading to more retractions.


Figure 2

Clear causal link

There is an adage in statistics that ‘correlation does not imply causation’. However, in this case the causal link is clear. If countries are increasing the number of published articles by more than 50% in five years, it is very unlikely that they will have increased their research spending at the same rate.

Good quality research requires expensive investment in equipment and the time needed to develop expertise. It is difficult to scale these up so rapidly.

The reason for such a prolific increase in publications is quite clear. Most ranking parameters and metrics value the quantity of publications. Without the requisite research infrastructure, researchers are forced to cut corners to fulfil their research targets.

In the light of this clear evidence, it is imperative that science evolves new metrics and measures which do not incentivise quantity and focus instead on quality research.

The scientific world does not need more bad quality papers to read, review, reject and retract. We have enough on our plate.

Achal Agrawal is founder of India Research Watch.

This article is a commentary. Commentary articles are the opinion of the author and do not necessarily reflect the views of University World News.