Research excellence – Beyond the buzzword

Centres of excellence and related initiatives have become a high-profile feature of the African science landscape during the last decade, where the eye-catching ‘excellence’ tag usually reflects either worthy aspirations or challenging levels of ambition. We believe that African science can and should take the concept of excellence more literally, moving from aspirational to ‘outstanding performance’ or ‘highest quality’.

Our recently collected empirical information on African science and opinions of its researchers provides novel insights on how this ‘performance upgrade’ could be achieved in the near future.

Interestingly the exact meaning of ‘excellence’ is left undefined in related policy documents or vision statements of excellence-promoting initiatives. It’s not difficult to see why: the underpinning concept of ‘research quality' is not easily pinned down, although its various meanings and implementations reflect a common notion that outputs and impacts of scientific research are (or have to be) of some relevance to others.

As we move into the arena of research assessment and evaluation, the meaning and measurement of excellence becomes even more blurred and contestable – especially with regards to gauging the external relevance of research in wider society, which is often beyond the reach of current analytical methods like peer review.

A harsh environment

Focusing on possibilities for identifying excellence within the world of science, we still face major obstacles in terms of operationalisation and quantification of quality. The key question is: what does excellence actually entail in Sub-Saharan Africa?

This can be a harsh environment, one where scientific research is often hugely under-resourced and aspiring scientists really struggle to perform at international levels of achievement. So it’s not very surprising to see that high-quality African research is often international collaborative research, in search of complementary financial and human resources.

In this context, what kind of quality standards should one then implement to divide the ‘best’ from the ‘rest’ in African science? Applying international quality standards is one option, which may also help create sustainable niches of excellence with researchers and facilities that can link up with global research networks and more easily attract resources. Another option is to impose other standards, better aligned to the development stage of African science.

This implies lesser selectivity in performance and more inclusivity in scope. Adopting such a ‘science for all’ model offers more opportunities for peer review panels to include assessment criteria and quality standards that also address socio-economic issues of local African relevance, rather than ‘Northern’ standards that tend to focus on publication productivity, citation impact scores, or research commercialisation achievements.

Excellence as multidimensional

The preliminary results from our survey to 74 researchers and research coordinators in various funding agencies, gave us some insights on the perceptions and operationalisation of research excellence in an African context.

One of the clearest indications is that excellence is perceived as multidimensional. While having publications remains one of the top-rated criteria to define excellence amongst researchers, other aspects, such as training young researchers, being an ethical researcher and producing work with great social and policy impacts, are equally considered central to the understanding of African research excellence.

When it comes to rating research proposals for funding allocation purposes, the highest consideration is given to the scientific rigour and a strong methodological basis, but also to the proposal’s potential for policy influence and social impact.

In this regard, it is interesting to note that researchers perceive that too much weight may be given to peer review scores, as compared to other criteria for the assessment of their research proposals.

Perceptions of excellence in research outputs slightly differ between researchers and research coordinators; while researchers consider that publications and social impacts are valuable measures of excellence in research outputs, coordinators give more importance to the commercialisation of research outputs and new technological developments.

Peer review panels are well-placed to provide value judgements of research proposals or outputs. Their assessment scores might lead to comparative categorisation on a measurement scale (‘weak measurement’), opening up the possibility for identifying top performers among individuals, projects and programmes.

But when it comes to ‘strong measurement’, the counting of research publications and citations is clearly one of the very few available options or systematic large-scale comparison. The survey respondents are also in favour of using such ‘bibliometric’ data as one of the information sources to identify excellence.

This endorsement raises the question why so few African researchers seem to embrace Google Scholar as a web-based platform to present their individual scientific achievements. This freely accessible information could open the way for large-scale transparency and (inter)national comparability of excellence. Of course, one needs access to the internet which is not always readily available in Sub-Saharan Africa.

Solving African problems

Apart from the scientific publication output and the citation impact of these publications, and easily captured information through Google Scholar profiles if researchers are willing and able to upload that information onto the internet, which other ‘dimensions’ of excellence are quantifiable within the African context?

Returning to the preliminary results of our survey we find that researchers and research managers are very keen to search for indicators that better capture the ability of research to influence policy and solve critical African problems.

Measures of research uptake by policy-makers and beneficiaries, indicators of collaboration, gender equity and youth promotion were mentioned as commonly overseen by mainstream research evaluation methods.

Research excellence is driven by individuals, but is also firmly embedded within their organisations. African research-led universities are the home of several niches of excellence. They act as attractors of research talent but also as organisational benchmarks within Africa.

Focusing on several of Africa’s largest research-led universities, our bibliometric data cover seven universities in South African and seven elsewhere in Sub-Saharan Africa – the University of Cape Town, University of the Witwatersrand, Stellenbosch University, University of KwaZulu-Natal, University of Pretoria, University of Johannesburg, North-West University, University of Botswana, Makerere University, University of Mauritius, University of Ghana, University of Dar es Salaam, University of Nairobi and Eduardo Mondlane University.

We identified their contributions in the top 1% most highly-cited publications per subfield of science worldwide during the years 1996-2015. An average 74% of those publications were internationally co-authored, with at least one research partner from outside Africa.

The University of Botswana is the only one among those 14 where more than 40% of their publications are without research partners outside the university. However, all universities have produced significant numbers of these single-university publications, thus signifying ‘home grown’ African research excellence.

Niches of excellence

So clearly, there are niches of international excellence on the continent. But Africa’s notions of research excellence should go beyond international research publications and their citations in the academic community.

Truly excellent researchers should be incentivised to move their knowledge and skills into the wider world, to generate wider impacts of Africa’s researchers in their communities – either in terms of science-based teaching and training, fundraising, networking, mobility and cooperation, commercialisation or innovation.

More suitable approaches to identify (potential) research excellence in African countries require experimentation. It is important to work towards offering peer review panels a broader selection of research quality standards and performance indicators that are adapted (or adaptable) to Africa’s research environments. Such a menu should, obviously, be supported by all major stakeholders including Africa’s researchers.

* This blog is based on two oral presentations by the authors at the Higher Education Research and Advocacy Network in Africa or HERANA III meeting on 22 November in Franschhoek, South Africa; and the Science Granting Councils Initiative's Annual Forum on 25 November in Maputo, Mozambique.

Robert Tijssen is professor of science and innovation studies. He is affiliated with the DST-NRF Centre of Excellence in Scientometrics and Science, Technology and Innovation Policy, or SciSTIP, in South Africa, as well as Stellenbosch University (South Africa) and Leiden University (Netherlands). Erika Kraemer-Mbula is senior lecturer at the Institute for Economic Research on Innovation, Tshwane University of Technology in South Africa, associate professor extraordinaire at Stellenbosch University, South Africa, and also affiliated to SciSTIP.