Of the various ratings of institutions idealists prefer the approach pioneered in Germany by the Centrum für Hochschulentwicklung or Centre for Higher Education Development - CHE. In 1998 the centre launched a well designed web tool of the its university ranking.

This is directed to students, allowing them to enter their preferences such as a law course with high teaching evaluations in a university with a strong sports programme. The tool shows all the programmes that meet the parameters entered by the inquirer, with each rated in the top, middle or bottom group on each of the criteria chosen.

Up to 22 criteria in seven domains can be nominated: teaching and learning, learning facilities including library and IT, international orientation, reputation, research and extra curricula.

In 2006, the Globe and Mail, the Education Policy Institute and the Strategic Counsel introduced a university report card navigator based on CHE's approach but I haven't found a current version. More recently, CHE extended its analysis to selected higher education institutions in Austria, Switzerland and the Netherlands.

Last year, the European Commission launched its ranking project inspired by CHE, inviting tenders for designing and testing a new multi-dimensional university ranking system covering institutions in and outside Europe, in particular the US, Asia and Australia.

In the first phase, until the end of this year, the winning consortium is testing its design with stakeholders. In the second phase during the first half of 2010, the consortium will test its design on a representative sample of at least 150 higher education and research institutions, focussing on engineering and business studies.

Idealists like CHE's analysis of institutions because it directs attention to the factors salient for students' choice of a programme and the institution where reliable information is available. It rates institutions within the margin of error the data support.

Because different students have different values, the CHE's analysis does not produce one universal league table but a different rank depending on the parameters entered and criteria chosen.

The CHE report card, however, downplays the fact that students, and indeed staff, do not choose their institution only on rational grounds. An important factor influencing many students' and staff's choice of their institution is its positional value.

The concept of positional value was introduced in The Social Limits to Growth (1976) by the International Monetary Fund economist Fred Hirsch. Many goods have positional as well as rational or objective value: real estate, restaurants, clubs and jewels, for example.

In higher education, positional value refers to the value that an institution confers on its students and staff in addition to the objective values of a good education and the route to a well-paying job. Some institutions, such as the members of the US Ivy League, the UK Loxbridge and Russell Group and China's C9 (see last week's UWN report here), have considerable social cachet in addition to their objective qualities.

Many students and staff put a lot of importance on an institution's reputation, status or positional value in choosing a place to study or work. While the CHE's multi-dimensional ratings include reputation, it does not produce the league table sought by those who value positional value highly.

One of the first world ranks of universities was the 'Champions league' of research institutions published in 2002 by the Swiss Federal Government's Zentrum für Wissenschafts und Technologiestudien, or Centre for Science and Technology Studies. The champions league ranks universities and other research institutions by their number and impact of research journal publications over a five year period.

The first world ranking of universities to gain international prominence was the Shanghai Jiao Tong University's institute of higher education's academic ranking of world universities. First published in 2003, this is still the most authoritative of the world university league tables - despite the by now familiar criticisms that it is narrow in ranking institutions only by research, it does not correct sufficiently for institutions' size, and it relies too much on data which is not sufficiently rigorous, such as Thomson Reuters' identification of highly cited researchers.

To those criticisms we may now add that institutions are starting to recruit Nobel laureates and highly-cited researchers not just because they extend the institution's research strengths but to increase their institution's ranking in the Shanghai league table. That is, the academic ranking of world universities is starting to breach Goodhart's law.

Charles Goodhart was a chief economic adviser to the Bank of England from 1980 to 1985. He first stated his 'law' as a jocular aside at a conference of the Reserve Bank of Australia in 1975. Goodhart's law has been formulated several times, perhaps best by the Cambridge social anthropologist Marilyn Strathern: when a measure becomes a target it ceases to be a good measure.

The Times Higher Education first published its in 2004 with data and analysis provided by the educational services company QS Quacquarelli Symonds. The THE and QS started with the laudable aim of ranking institutions on their teaching and overall reputation in addition to their research performance.

Unfortunately only a few countries have indirect indicators of teaching quality and there is no international indicator in higher education. A large proportion, 40%, of the THE-QS league table was based on the QS survey of academics' views of the standing of the institutions they claim to know. Response rates are low, there is no convincing demonstration that the sample actually represented the academic population and the data are not made available to be verified independently.

There are now several world university league tables based on various measures of research publications and citations: the Higher Education Evaluation and Accreditation Council of Taiwan's performance ranking of scientific papers for world universities, first published in 2007, Leiden University's Centre for Science and Technology Studies' first published in 2008, and SCImago institutions rankings: world report first published in 2009.

Leiden's world ranking is useful because it is normed for institutional size which makes research quality more important than research volume. SCImago's institution rankings is also useful because, unlike the other world university league tables, it does not rank just the top 200 or 500 universities.

SCImago ranks all institutions which had at least 100 publications indexed in the Scopus publications database in 2007 and therefore includes 1,527 higher education institutions. Even so, it is highly selective and includes only 16% of the world's estimated 9,760 universities.

The more comprehensive coverage of SCImago's institution ranking makes it possible to compare several countries' performance. There seems to be two patterns among countries whose research universities have a high average volume and quality of research publications.

One group has a fairly even performance of their research universities, presumably because they have had fairly even levels of government support. This is in northern continental Europe and includes Switzerland, Germany, Sweden, the Netherlands, Austria, Denmark and Finland.

The other group also has a high average volume and quality of research publications but is spread much more unevenly between universities. This group includes the US, UK and Canada.

Earlier this year, an independent Russian rating agency RatER used a pool of experts to rate more than 400 universities on seven scales: academic performance, research performance, faculty expertise, resources, socially significant activities of graduates, international activities of the university, and a survey of expert. The resulting league table is called the global university ranking.

Again, this rank depends heavily on the judgement of the experts chosen by RatER and to gain credibility it will have to disclose more information on who the experts are or at least how they were chosen.

Quite a different approach has been adopted by the École nationale supérieure des mines de Paris' international professional ranking of higher education institutions established in 2007. This table ranks higher education institutions by the number of graduates who become chief executives of Fortune Global 500 companies, the 500 companies with the biggest revenue.

This is rather a narrow criterion upon which to rank institutions. One would also want some demonstration that graduates became chief executive officers of Fortune Global 500 companies because of the education they received rather than because of the ability, cultural capital and connections that gained them admission to the highly selective institutions included in the ranking.

Demand for world university league tables will last for as long as large numbers of students study internationally. All world university league tables have limitations or flaws, and many have both. There should therefore continue to be changes in the league tables until there is a reconciliation of the expectations of them and the rankers' capacity to meet those expectations.

Readers can keep up to date on changes to league tables and the considerable literature on them through the ranking systems clearinghouse. This is maintained by the Institute for Higher Education Policy, an independent, non-profit organisation based in Washington, with funding from the Lumina Foundation for Education.

Gavin Moodie is a policy analyst at Griffith University, Australia.

Printable version
Email to a friend
Comment on this article

Disclaimer: All reader responses posted on this site are those of the reader ONLY and NOT those of University World News or Higher Education Web Publishing, their associated trademarks, websites and services. University World News or Higher Education Web Publishing does not necessarily endorse, support, sanction, encourage, verify or agree with any comments, opinions or statements or other content provided by readers.