The consortium running the European Commission-backed U-Multirank is working to ensure the quality and reliability of the indicators it will use to measure universities, University World News has been told in an exclusive interview.
Jordi Curell, director of higher education and international affairs in the commission’s directorate general for education and culture, said assessments were weakened by universities failing to monitor the professional success of their graduates.
“There is an issue because some of the indicators are difficult to obtain, for instance when it comes to the tracking of students, because they don’t exist everywhere,” he said of the new global ranking system that was officially launched in Dublin in January.
U-Multirank aims to step out from the crowd of international university rankings by focusing not only on institutions’ research quality, as other rankings largely do, but also measuring teaching and learning, knowledge transfer, and the international orientation and regional engagement of participating universities.
Some of these, such as the quality of teaching, are more difficult to measure, admitted Curell. “That’s done in particular through student surveys, but also in terms of completion rates, dropouts,” he explained to University World News.
“Other ways to quantify the quality of teaching and learning are by measuring the staff-to-student ratio, and percentage of students graduating within the norm period.”
Moreover, students in different countries do not rate their expectations of a university in the same way.
The organisations running U-Multirank therefore “have techniques to make sure that the results of surveys of students in different member states or beyond are comparable by avoiding national biases because of the different level of expectations, for instance,” said Curell.
He is confident that the challenges will be overcome, since the institutions running this new ranking system have run rankings in different European Union (EU) countries, and their professional and research credentials are well proven.
The consortium running U-Multirank is led by the German-based think-tank the Centre for Higher Education (CHE) and the Centre for Higher Education Policy Studies (CHEPS), an interdisciplinary research institute in Twente University's school of management and governance in The Netherlands.
Other partners in the consortium are the Centre for Science and Technology Studies at The Netherlands’ University of Leiden; Elsevier, the Dutch academic publishing giant, which is providing scientific, technical and medical information products and services; German publisher the Bertelsmann Foundation; and German software provider Folge 3.
In the meantime, the drive for universities to participate in the first edition of U-Multirank has gathered about 320 universities so far, with 180 still needed to reach the target of 500 that the consortium has set to achieve sufficient critical mass.
Universities can still join until early autumn if they want to be included in the first ranking results, set to be published in the first half of next year.
Gero Federkeil, the manager in charge of rankings at Germany’s CHE and vice-president of the Warsaw-based Observatory on Academic Ranking and Excellence (IREG), has encouraged universities to participate, saying it would make them more visible to students seeking to study and will show how they perform in comparison with very similar institutions.
“There are no composite indicators, no pre-defined weights on individual indicators, [just a] single ranking for each indicator,” Federkeil said during an event debating U-Multirank held on 30 April in Brussels.
According to Federkeil, the website will give students the ability to identify a sample of comparable universities or examine the profile of a particular participating university and see how it fares in comparison with similar institutions.
The website is currently being designed and will be accessible when the first round of results is ready next year, Curell told University World News.
It will provide a set of institutional rankings, looking at how a university performs through all the major indicators set by U-Multirank.
Initially, four academic fields will be ranked: mechanical engineering, electrical engineering-information technology, business-management and physics.
The second round of rankings, which is scheduled to be published at the end of 2014, will aim to cover at least 625 universities and to also rank universities providing degrees in computer science-IT, sociology, psychology, music and social work.
The definition of U-Multirank as a ranking system has been questioned since it refuses to provide a league table of universities in Europe and around the world.
“We stay in the ranking terminology to question the traditional rankings explicitly,” explained Federkeil, noting that the marketing and communication regarding this new system will make clear the difference between it and other rankings.
Even though U-Multirank has been criticised in the UK's House of Lords parliamentary chamber in 2012 and other players in university rankings, Curell is optimistic that the EU-backed system is on the right track.
“Of course, it will improve with time, it will evolve with time, but so far there is no reason to be worried. On the contrary, I think there’s only reason to be optimistic,” he said, underlining that he was confident about the results to be delivered by U-Multirank.
Receive UWN's free weekly e-newsletters