UK: League tables in the spotlight

Around the world, university league tables influence students, academics and governments. Foreign students increasingly use them to select an institution in another country while governments and scholarship bodies use them to inform decisions about supporting students. Then there are the academics who use them to decide where they want their next job. Yet questions remain about the quality, methodologies and robustness of the rankings, as a new report notes. Counting what is measured, or measuring what counts? was commissioned by the Higher Education Funding Council for England.

The report's purpose is to stimulate informed debate about league tables across the higher education sector and not to endorse any particular approach, explains Professor David Eastwood, the council's chief executive: "We certainly do not intend to introduce an official published ranking, as some have suggested. As a funder of higher education, we have an interest in ensuring that the sector is accurately presented to prospective students, policy-makers and others with a stake in the quality of higher education; and that the relative strengths of particular institutions are appropriately recognised and reflected."

The 64-page report analyses world rankings by Shanghai Jiao Tong University Institute of Higher Education and THES-QS, and national league tables published by The Sunday Times, The Times and The Guardian.

"The influence of league tables is increasing both nationally and internationally, and cannot be ignored despite serious methodological limitations. They are being used for a broader range of purposes than originally intended, and are being bestowed with more meaning than the data alone may bear," warns the report.

Given this influence, there is an onus on policy makers and institutions themselves to promote greater public understanding of league tables and alternative sources of information about higher education, the report says. It recommends that governments internationally should cooperate to codify good practice for both compilers and users of league tables.

The researchers found the rankings of the most influential league tables of British institutions largely reflect reputational factors such as entry qualifications, level of degree attained and Research Assessment Exercise grades. Their analysis indicates that measures used in the tables are largely determined by the data available, not by clear definitions of quality; that some of these measures are poor indicators of the qualities identified; and that methods for calculating scores are not always transparent, and some produce non-standardised results.

The report says institutions are strongly influenced by league tables, although many of them are reluctant to acknowledge this. Tables are used as key performance indicators and by senior managers and governors to drive internal change; some universities believe that rankings will become more influential as higher education becomes more competitive.

In the analysis, researchers from the Open University's Centre for Higher Education Research and Information (Cheri) and Hobsons Research found the five league tables did not provide a complete picture. They focused on full-time undergraduate provision and institutional rather than subject-based rankings which resulted in the exclusion of a wide range of specialist, postgraduate, small or predominantly part-time institutions from the published tables.

There was insufficient transparency about the way the league tables were compiled: methods for calculating the scores for each institution were not always made clear and rankings reflected reputational factors, not necessarily the quality or performance of institutions. The format and content of the tables could be brought up-to-date by being made more easily accessible, interactive and reflecting recent developments in online learning, says the report.
Full report on the HEFCE site