EU university ranking may ‘harm’ higher education

A key international advisory group for UK higher education has warned that U-Multirank – the European Union’s (EU) multi-million euro alternative to commercial international rankings – may harm rather than benefit the sector.

But the U-Multirank team of Frans van Vught and Frank Ziegele quickly issued a statement seeking to refute the UK HE International Unit’s analysis.

“As we think the concerns lack validity, we strongly reject the conclusion,” they said.

In a briefing note to UK universities, the International Unit (IU) says that concerns have been raised that public funding lends legitimacy to U-Multirank, and performance as judged by the tool could become the basis for future funding decisions.

U-Multirank is also open to criticism for relying at least partially on self-reported data, which lacks verifiability, it adds, and the IU relays fears that the multi-dimensional tool brings together “incommensurate” variables that are combined to create a league table, thus undermining efforts to go beyond rankings.

U-Multirank is to receive €2 million (US$2.6 million) from the EU’s Lifelong Learning Programme to support its development in 2013-14, with the possibility of a further two years’ funding after that.

The IU notes the view that the money could be better spent on other EU priorities.

The EU’s ranking was launched in January and will eventually provide an institutional ranking of whole institutions as well as field-based rankings initially for engineering, business and physics.

Data collection is due to begin this month or possibly in June. The first phase of its development will focus on 150 pilot institutions and gathering individual expressions of interest.

First results are expected to be available in February 2014. The aim is for 500 institutions from across Europe and the world to take part. A consultation process on the refined indicators and the web tool will continue in parallel with implementation.

The League of European Research Universities, LERU, which represents 21 research-intensive institutions across Europe including the universities of Cambridge and Oxford, University College London, Imperial College London and the University of Edinburgh, formally withdrew its support for the project in January.

Secretary General Kurt Deketelaere called the scheme “at best an unjustifiable use of taxpayers' money and at worst a serious threat to a healthy higher education system”. The IU says it seems likely that a number of ‘leading’ universities will not take part.

But Androulla Vassilliou, European commissioner for education, training, youth and sport, said at the January launch in Dublin that the EU was “best placed to help the different stakeholders to take a broader view of their performance, and create transparency beyond the national systems.

“[U-Multirank] will allow an institution – but also departments within institutions – to benchmark itself against its peers across borders, for many, for the first time ever. And not only against its peers in Europe, but in all quarters of the globe.”

Response from U-Multirank

On the issue of EU funding, the U-Multirank team points out that it is limited to four years. Funding "is provided on the clear condition that it evolves into an independent and self-funding initiative. The legitimacy of the whole system can only come from independence, not from public funding.

"The funding of U-Multirank represents 0.02% of the funding spent on higher education through the Lifelong Learning Programme – a programme which in the current funding period constitutes less than 2% of the total EU budget."

Rejecting the suggestion that U-Multirank could be used to influence funding decisions in future, Van Vught and Ziegele say: “The European Commission has already made clear that it will not use the results of the ranking for any funding decisions – and also that it opposes the use of other higher education rankings for such purposes.

“The multi-dimensional character of U-Multirank makes it extremely difficult in any case to base funding decisions on it, unlike the existing global rankings, which – due to their simplicity – trigger a ‘reputation race’, frequently leading to costly national and institutional funding decisions in order to gain a few places in the rankings and so move up the scale of reputation.

“U-Multirank, on the other hand, will lead to a situation where differentiated ranking data will be used for strength-weakness analyses, instead of simplistic connections of performance and funding.”

Challenging the view that the rankings marketplace is already overcrowded, they say: “This innovative approach to ranking will be more than ‘just one more ranking’ – in particular with regard to the benefits it offers to universities to compare and benchmark, and to mobile students supporting their choices.

“In its 2013 ranking report, the European University Association cited U-Multirank as being ‘substantially different from existing global rankings’ and we could not agree more.”

And Van Vught and Ziegele say the suggestion that U-Multirank will draw on “incommensurate data” to create a league table is based on an “unfortunate misunderstanding”.

“U-Multirank will not create league tables. This is one of the basic methodological principles which was already outlined in the feasibility study and report as well as in our bid for the implementation project.

“However, the IU comment could be better directed against traditional rankings, which do indeed mix up very different aspects of performance into a single composite overall score. Instead U-Multirank will deliberately avoid this by presenting multi-dimensional profiles in which a university can perform well on some indicators and less well on others.”

The full response to the IU’s criticisms can be seen here.