Pressure on rankings may lead to a more meaningful exercise

Is this the beginning of the end? Over the last years, news reports and blogs have been filled with stories and commentary articles about university rankings and their future. Some ask if this could be the slippery slope towards their demise while others are more sceptical.

What’s happening?

In 2020, the Chinese ministries of education and science and technology renounced their reliance on the Science Citation Index for academic assessment, appointment and promotion. In a move similar to actions by Dutch universities and funding agencies, the changes marked the intention to develop an evaluation system that is more reflective of Chinese context and values.

In May 2022, three Chinese universities reported they were quitting international rankings.

The Russian invasion of Ukraine provoked a sequence of actions aimed at cutting Russia off from the global scientific community. Following publication of the Russian Union of Rectors’ statement supporting the invasion in March 2022, the European Union, the European University Association and others ceased collaboration with their Russian counterparts.

U-Multirank, Times Higher Education (THE), QS and Webometrics variously announced their intention to reduce their coverage of Russian universities; although by April, QS had modified its position.

In response, Russia announced it was dropping out of Bologna and would design a new system to meet “national interests”.

Around the same time, in the United States, several well-publicised scandals exposed rankings’ unsavoury underbelly. The Operation Varsity Blues bribery scandal saw 33 ‘famous’ parents of college applicants accused of paying more than US$25 million, while others were accused of fabricating sports credentials and entry requirements. Convictions and jail terms followed.

In 2021, the former dean of Temple University’s business school, along with two co-conspirators, was convicted of fraud for falsifying data provided to US News and World Report (USNWR). Next, Columbia University was accused of “cheat[ing] its way to the top of the US News rankings”, while Rutgers University was charged with allegedly manipulating graduate outcomes.

In November 2022, several US law schools – led by Yale – announced their withdrawal from USNWR because the methodology used “flawed survey techniques and opaque and arbitrary formulas”.

Beyond the headlines, there seem to be two trends with intersecting strands.

Geopolitical shifts

Before the COVID-19 pandemic and Ukraine war, geopolitical tensions were mounting.

Brexit (2016) took UK universities outside the world’s largest research and innovation programme (€95.5 billion or US$101 billion) – which recent changes in UK quality assurance threaten to do for teaching and learning.

Events have destabilised global production and supply chains and exposed relative weaknesses in ‘Western’ research and innovation systems. Economic interdependence, the hallmark of global expansion and multilateralism, shows signs of unravelling.

Global rankings have tracked China’s steady ascent as a science and technology powerhouse. China (mainland) achieved eight universities in the Academic Ranking of World Universities (ARWU) top 100 in 2022 compared with none in 2003 – and the United States dropped from 58 to 39 universities. More significantly, China had 71 in the top 500 compared with 19 in 2003, in contrast to the US which declined from 161 to 127 universities.

In contrast, Russia’s ambition has been woefully less successful despite coming from a stronger scientific tradition and investment through its Project 5-100 programme. No university ranked in the ARWU top 100 and only two ranked in the top 500 in 2022.

Whatever one thinks about rankings, they have helped fashion a global higher education and scientific system. The annual visualisation of ranked universities in which a growing number of global players – competitors but also collaborators – jockey for position has transfixed and transformed policies and strategies around the world.

How soon before the rankings reflect recent developments? For different reasons China and Russia may have decided there is little more to be gained from participating in rankings.

Countries and universities which rely on international students may have a different perspective, however. As for rankings themselves, these events are likely to foster further expansion of their different rankings and consultancy services customised to different audiences and new geopolitical constituencies, such as the MENA (Middle East and North Africa) region and Africa.

Equity, values and new priorities

If geopolitical shifts are unlikely to significantly affect global rankings, did COVID shine a light on other priorities?

Social inequities and regional disparities in educational and life opportunities and outcomes preceded the pandemic, but they were accelerated by it. There is pressure for a more responsive education system to help solve societal challenges, focus on equality, diversity and inclusion, improve student success and undertake impactful research and innovation.

In such circumstances, it’s hard to defend the behaviour of universities which prioritise being ranked among the top 100. That rankings measure the outcomes of historical advantage in the era of mass higher education is a political embarrassment.

US Education Secretary Miguel Cardona called rankings a “joke” which did “little more than Xerox privilege”. Comments like this may explain why some US law schools suddenly changed their minds.

The pace of grassroots initiatives challenging the status quo has quickened in recent years. Since 2005, the Washington Monthly's College Guide and Rankings has led the way, measuring college and university contribution to social mobility, research and providing opportunity for public service.

The Economic Mobility Ranking examines colleges which enrol the highest proportion of students from low- and moderate-income backgrounds and provide them with a strong return on their educational investment.

The Center on Education and the Workforce has pioneered a return-on-investment ranking of 4,500 US colleges. The US and UK have both developed a Social Mobility Index Ranking.

Alternative rankings also seek to measure academic freedom and integrity and sustainability. The early mover was UI GreenMetric, initiated by Universitas Indonesia in 2010.

Globethics.net University Ranking proposes to use student and staff feedback to measure university contribution to building just, peaceful, sustainable and inclusive societies.

The Center for Open Science ranks academic journal commitment to research transparency and reproducibility.

The Academic Freedom Index focuses on academic freedom, campus integrity and freedom of academic and cultural expression.

The big players have been compelled to respond. USNWR now includes social mobility information and is changing the way standardised tests are scored. THE and QS have both capitalised on the popularity of the Sustainable Development Goals, introducing their own impact or sustainability rankings.

The challenge for innovators lies in their ability to identify meaningful indicators and data. This is much more difficult than may appear. Rankings are resource intensive.

Redefining excellence

Global rankings, and USNWR, are unlikely to disappear soon. Neither the actions of China and Russia, nor of US law schools, are likely to undermine the recognition value that rankings bring to countries and institutions eager to strengthen their presence in global science or the world economy. As Morocco’s success in the FIFA World Cup illustrates, nothing beats being on the world stage.

Moreover, the big players have shown themselves to be adaptable, substituting institutional data with other sources. They’ve also learned to monetise higher education data, partnering with publishers and data analytics. Today, they are primarily a vehicle for collecting and selling data.

Yet, changes are afoot. The abovementioned decisions to shun rankings can be interpreted as defiance of prescribed categories of excellence. These same forces are evident in grassroots efforts to redefine excellence beyond concentrating on citations and indicators that reinforce historic advantage and prestige.

Emphasis on equity, ethics, integrity and sustainability are in tune with today’s priorities. Over time, these small actions might have a death-by-a-thousand-cuts effect on rankings – leading ultimately to a more meaningful exercise.

Ellen Hazelkorn is joint managing partner, BH Associates, and joint editor, Policy Reviews in Higher Education. She is co-editor of Research Handbook on University Rankings: Theory, methodology, influence and impact.