The role of rankings in improving higher education

This month the European University Association published a significant piece of research that highlights one of the unintended consequences of university rankings. It validates the activities of national and international ranking agencies by recognising the role they play – or potentially play – in the process of the continual improvement of higher education.

I say ‘unintended’ because at the very beginning of the league table era, back in the 1990s, few people would have expected universities to recognise the relevance of rankings to their operations and implement changes as a result.

In the United Kingdom, vice-chancellors actively sought to strangle the embryonic Times national university rankings at birth, and this hostility at senior level permeated the academic profession across Europe.

A decade later, when I was involved in the early discussions that led to the establishment of IREG – the Observatory on Academic Ranking and Excellence – university rankings were still widely viewed with deep suspicion by the academic community.

As we can see from findings of new research published by the European University Association, some doubts remain.

The survey of 171 higher education institutions from 39 countries is outlined in a report, Rankings in Institutional Strategies and Processes: Impact or Illusion?, authored by Ellen Hazelkorn, Tia Loukkola and Therese Zhang.

But few universities say that they ignore the rankings. According to the survey, 71% were either already using the results of rankings to take strategic, organisational, managerial or academic action – or were planning to do so.

Backdrop

But academics and university administrations are not and never were the target audience for rankings. Institutional improvement was never the goal.

Rankings began as – and remain – a source of information and guidance to help prospective students through the increasingly complex process of selecting a university and a course that meets their requirements.

An important health warning here that is reflected in the report and accepted by all ranking agencies: they are one of a number of sources of information and should not be seen as the sole factor in guiding student choice.

But they are an important one, especially, I would argue, the national rankings on which the growing range of international rankings – Shanghai’s Academic Ranking of World Universities, QS, Times Higher Education and now US News – have built.

My involvement in the UK was initially with a national ranking, working with John O’Leary and his colleagues on the pioneering Times national university rankings, which used verified statistics, mostly collected from the universities themselves by national agencies, as proxies for the elusive properties of academic quality and effectiveness of process.

I now work with another, The Complete University Guide, which is independent, entirely web-based and completely free to access. It is not associated with any specific media operation. The Guide’s site includes subject rankings covering some 67 academic areas so that prospective students can quickly find the leading universities in their chosen discipline.

There are detailed university profiles, guidance for applicants, careers advice, sections for international students – incoming and outgoing – advice on housing and finance, distance learning, postgraduate study and sports facilities. Also, controversially, there is a ranking of universities by the level of student-relevant crime within five kilometres of the main campus.

The UK has been fertile ground for rankings.

Prospective students were successively faced with a large number of new universities with unfamiliar and sometimes confusing titles following the redrawing of the higher education map in the early 1990s, and almost two decades later, the politically contentious shift in 2012 to considerably increased tuition fees of up to €11,400 (US$14,300).

This second development has had the predictable effect of focusing minds on the value for money that students could expect to receive while at university and their chances of a good job on graduation.

Rankings remain receptive to criticism and, speaking for The Complete University Guide but in the knowledge that others act in a similar fashion, we engage closely with the academic community to improve our methodology and presentation.

It is critically important that our methodologies are transparent and logical, that they change to reflect the developing higher education environment, and that they are easily accessible to young people who may not be practised in accessing providers of information such as U-Multirank and the UK’s state-backed KIS.

The response of universities

We are aware, and have been for some time, that universities watch our activities like hawks, whatever they may say in public.

There is enormous interest in the timing of the annual publication of the main rankings, and after publication a deluge of requests from marketing departments for explanations for their university’s climb, or more often fall, in the rankings.

But we are also aware that universities seek to maximise their position in the rankings. This can involve some alteration in a university’s data collection or the re-assignment of a particular activity or expenditure to one budget head or another.

It is great from a ranking organisation’s perspective to see that the vast majority of universities monitor their performance – and a matter of some surprise that 6% say they do not. It is also good to see that 63% feel rankings have had a positive influence on their reputations and only 7% a negative one.

But for me the critical finding is that 71% make use of rankings to take strategic, organisational, managerial and academic decisions. I only hope that, as the report also emphasises, other sources of information are used too…

If these decisions lead to a better experience for a university’s students, the intervention of league tables has had the desired effect.

This could be in terms of a better understanding of its declared mission, more or better targeted tuition, improved careers advice, or better facilities for students and academic staff.

But if a university’s actions are limited to book-keeping exercises, designed to massage data for a better position, then an opportunity is lost and that institution has betrayed the trust students – and the community – have placed in it.

* "Rankings in Institutional Strategies and Processes: Impact or Illusion?" by Ellen Hazelkorn, Tia Loukkola and Therese Zhang, is available for download at the European University Association website.