SOUTH AFRICA: Researcher rating boosts publication

Recent research indicates that the evaluation and rating system for individual researchers by the National Research Foundation (NRF) has had a positive impact on the publication profile of South Africa's researchers in the social sciences.

A paper by Roula Inglesi-Lotz and Anastasious Pouris, published in the September edition (Volume 88, number 3) of Scientometrics, indicates that NRF rating of researchers increased by an estimated average of 24.5% the number of research articles produced by social scientists in the first five years after the rating system was opened up to social scientists in 2001.

The incorporation of the social sciences into the NRF system, initiated by the publication of the NRF Act of 1998, provides a unique opportunity to investigate the effect of NRF programmes on research performance in the social sciences, say the authors.

The ratings system is made up of five categories that acknowledge both established and emerging researchers.

The top 'A' rating is rewarded with research funding of R100,000 (US$14,000) per year for as long as the recipient retains his or her rating. In order to be favourably assessed by the NRF, scholars need to have a considerable number of publications in internationally refereed journals.

As the authors note, many of the country's research-intensive universities have incorporated the rating system into their own promotion and remuneration policies as an extra incentive for researchers. The ratings and the numbers of academics holding ratings are frequently publicised by universities seeking to market the quality of their institutions.

Titled "Scientometric Impact Assessment of a Research Policy Instrument: The case of rating researchers on scientific outputs in South Africa", the paper by Inglezi-Lotz and Pouris argues that research and policy administrations can use individual rating and evaluation as a tool to improve research domains.

But the authors, who are both at the University of Pretoria, stop short of claiming that the estimated impact on social science research would apply across the board in scientific disciplines.

"For example, clinical medicine [not supported by the NRF] researchers may already publish the majority of their research abroad, and hence they would not have the opportunity to switch their publications from local to international journals as researchers in other disciplines can do," they write.

Using scientometric information, the quasi experimental 'before-after control impact' method and the econometric breakpoint test as developed by economist Gregory Chow, Inglesi-Lotz and Pouris argue that the NRF system, supported by university administrations, provides continuous incentives for researchers to become prolific and publish in international, Institute for Scientific Information-indexed journals.

They suggest that as it became public knowledge in 1999 that the NRF was to become responsible for supporting social sciences as well as traditional scientific disciplines, social science researchers started publishing in international journals and increased their productivity.

"Within two years, the additional publications were published in the international literature (2001) and the discontinuity became detectable," they write.

While such quasi experimental methods for policy evaluation are at a very early stage in South Africa, they constitute a "new powerful set of tools for empirical analysis and evaluation of government and agency policies" which are used extensively in the United States, Canada, the United Kingdom, some European countries and within international agencies such as the World Bank, say the authors.

The findings, they argue, are supported theoretically by behavioural reinforcement theory according to which human behaviour is determined to some extent by environmental factors - in this case incentives, which act as motivators for researchers to improve their research profiles by publishing more articles and in internationally recognised journals.

Inglezi-Lotz and Pouris argue that in the South African context the NRF rating system has the potential to become a powerful policy instrument providing incentives for researchers to remain in the country or to foreign researchers to move to the country.

"In addition, the instrument will provide incentives to researchers to aim towards excellence and will contribute towards making the academic profession a desirable objective for students."

Citing 2007 research by Habib and Morrow, they emphasise that the culture of publishing is not prevalent across all South African universities and research institutions, for reasons ranging from inadequate remuneration and working conditions to poor management performance in the knowledge system.

Because of this it should be expected that the NRF system "will have an impact on researcher outputs until the researchers' productivity reaches its limits or the researchers are not sufficiently motivated by the offerings of the system".