For generations of scientists, ‘peer review’ has been the cornerstone in advancing science and securing healthy scientific development. It is what distinguishes a scientific publication from any other publication and is the foundation of citation- and bibliometrics-based quality systems.
The methods peer review uses can be criticised for often lacking structure and transparency (and there are major differences between scientific cultures).
However, the review usually concerns a limited number of research projects within a defined field, a limited number of researchers on a defined position, or a single article for publication, and therefore can be said to mostly function well and serve its purpose.
It builds on the reviewers’ experience and excellence and on their responsibility to advance their scientific fields. Independent reviewers are often anonymous so they do not hold back on giving a severe critique when this is necessary.
But we all know that ‘advancing a scientific field’ also can include toxic battles between different traditions and research groups and that a single review may not always be the most neutral and objective assessment.
In addition, therefore, citation and bibliometrics systems have grown and developed that have got their own flaws, but are nevertheless based on opinions of what is relevant and useful ‘new knowledge’ from a much larger part of the scientific community than just single reviewers or small review teams.
Attempted hijack of peer review concept
Swedish politicians and administrators recently attempted to hijack the peer review concept, and their proposals would make peer review more synonymous with an open and casual discussion between reviewers and reviews, with poorly defined criteria for what is good and bad. They propose site visits and personal meetings that will affect evaluators’ judgments.
Scientifically illiterate politicians and administrators propose to use these opinions to distribute funding, and praise this method as a means of preserving universities’ autonomy. In reality it will become a tool allowing them to govern the development of research.
Reviewers can give valuable advice to their colleagues on the research and teaching they appraise. But the peer review method is completely unsuitable and unreliable for the use of allocating financial resources on a national level.
Uppsala University recently did a peer review of its research – which accounts for about 10% of Swedish academic research – and had to divide it into almost 89 fields in order to find suitable experts.
Upscaling this to the national level would be a truly costly affair – approximately €10 million (US$13 million) not counting the cost of self-assessment work – if it is at all possible.
In order to create some sense of equality and fairness, the results would then need to be normalised – within and between different fields, from the humanities to engineering. The problems of field normalisation for bibliometrics are difficult enough, but how could this possibly be done when it is to be based on thousands of unstructured peer review reports?
In addition, the review teams would feel massive pressure, not only from the colleagues they are evaluating but also not to destroy the funding base for their own fields globally. Such pressures could clearly be seen in the early Swedish discipline evaluations during the 1980s and 1990s.
Reviewing will be transformed into a social game and not a fair evaluation based on actual achievements. For researchers and teachers, evaluations will appear as utterly untransparent and unfair.
A very likely development is that individual groups will start using citations and bibliometrics (now freely available) on their own. But at what cost for transparency and fairness? A similar story seems to be on the cards for Swedish higher education quality assurance.
Some review teams solely evaluate students’ written bachelor and master theses (‘results’) and some base their evaluations on self-assessment exercises (‘processes’). The result is that no one knows why certain decisions have been made.
Well-known, high quality programmes have had their degree-awarding powers questioned and all background materials are hidden deep down on a website with any methodological problems passed over by the relevant authority, the National Agency of Higher Education.
The difference for research is that there is no European Association for Quality in Higher Education to save us.
The minister of education and his under-secretary – neither with any experience of academic research or teaching – state that this ‘peer review’ system will improve Swedish research.
We find this hard to understand. How can a less competitive, less transparent and unfair reviewing method make this happen? Quality assurance should reward the best researchers among all peers, the Messis and Ibrahimovichs of science.
Swedish quality assurance of academic education and research has become a replica of Hans Christian Andersen’s fairytale about the emperor’s new clothes – they do not exist.
The risk of creating total havoc in the system is imminent. We are astonished that this has been allowed to happen. We are equally astonished at the lack of open debate, critique and action from the Swedish scientific community.
In our opinion, the best solution would be to discard national quality systems and give universities real autonomy to handle their external quality assessments themselves by accredited global quality agencies.
This would truly promote the quality of academic education and research and additionally save large amounts of Swedish taxpayers’ money now spent on meaningless charades.
* Lena Adamson is associate professor of psychology at Stockholm University. She is also a contracted expert for the European Institute of Innovation and Technology (EIT)-European Commission, on quality assurance, teaching and learning for creativity and innovation, and a consultant for the Council of Europe on quality assurance in higher education. Anders Flodström is professor of material physics at the Royal Institute of Technology. He was rector of Linkoping University from 1996-99 and of the Royal Institute of Technology from 1999-2007, and University Chancellor of Sweden and head of the Swedish National Agency for Higher Education from August 2007 to June 2010.
Receive UWN's free weekly e-newsletters