The return of the (revised) Teaching Excellence Framework

The wicked problem of measuring teaching quality has not been solved in the new, revised Teaching Excellence Framework (TEF), which also means the methodological improvements that have been made may not be enough for the TEF to fend off its fiercest critics.

Once again work is underway to assess the performance of the teaching mission of UK universities. The Teaching Excellence Framework, the results of which first appeared in 2016, has returned following revisions.

In January, all higher education providers with over 500 students made their submissions to the national scheme run by the UK’s Office for Students. This written submission is considered alongside a set of indicators, which includes responses to the National Student Survey and metrics on student continuation and completion. This evidence is assessed by TEF panels, comprised of academics with leadership responsibilities and students with experience of representing their peers.

Before July, submissions will be assessed and a provisional rating for each provider will be formed. This rating, and the reasoning behind it, will then be shared with the provider, enabling them to make representations before the final ratings are decided.

Then, in September 2023 the submissions and outcomes of the TEF will be published. We can then expect to see providers displaying their new rating which will last for four years. As before, these will be Olympic-inspired ratings of gold, silver or bronze; however, this time there will be a new rating of “requires improvement”.

The TEF has followed its own learning curve, having been revised since we last saw it a few years ago. This time providers will receive an overall rating as well as two underpinning ratings – one for student outcomes and another for student experience – to signal where a provider excels in one aspect.

Another new feature is the independent student submission. This puts a student voice, of sorts, directly into the TEF process. This is designed to provide insights into what it is like to be a student at a particular provider. Although this component is optional, 204 student submissions from 228 participating institutions were received.

A wicked problem

Although the TEF methodology has been revised, the perennial problem of measuring teaching quality remains. For university teaching, unlike research, there is no consensus around a definition of excellence or how to measure it. This, in part, explains the dominance of research measures in the well-known global rankings.

As Philip Altbach and Ellen Hazelkorn note, it is impossible to adequately assess education quality for the purposes of international comparisons.

And, as my book Teaching Excellence? Universities in an age of student consumerism reveals, measuring university teaching quality within just one country is very challenging.

A great deal of the methodological work undertaken to measure university teaching in new ways is on the cutting room floor. Several of the new metrics developed for the TEF – such as Teaching Intensity and Learning Gain – do not feature in the new scheme. This is because they did not work as planned. The wicked problem of measuring teaching quality has not been solved.

This also means the methodological improvements may not be enough for the TEF to fend off its fiercest critics.

A blunt tool to inform choice

The original TEF had two overarching roles: first, enhancing provision by incentivising excellence, and second, providing information to help applicants make their choice about where to study. However, the new generation TEF is mainly about the first, with the second relegated to a minor role.

The new TEF is not expected to contribute to the “food labelling of degree courses” – providing information akin to the ingredients and nutritional information we find on what we buy to eat and drink.

This is because those running the scheme are conscious of the fact that TEF results may be misleading to applicants, particularly if considered in isolation. For example, an applicant may believe their graduate outcomes would be better if they chose a TEF gold institution, when this may or may not be the case.

Moreover, the more granular subject-level TEF is not being pursued, leaving only the provider level ratings which are a blunt tool to inform choice. TEF results are, therefore, summaries for providers as a whole, generated by extensive number crunching of various metrics involving benchmarking. This means they are perhaps more like the ratings of supermarkets rather than food labels.

Marketisation

The new TEF is also detached from the agenda of making undergraduate education in England more of a free market. Its predecessor was introduced to provide information on which courses were excellent, so applicants could make an ‘informed choice’ on where to study.

This was to help applicants navigate the now more liberalised marketplace – which includes new entrants such as private and for-profit providers which had been welcomed in by the government.

The idea was to unleash the discipline of competition and choice, although, in practice, the difficulties of measuring teaching quality also apply to generating product information. In other words, if we are not sure what teaching excellence is and cannot really measure it, then we cannot tell people where it is.

Considering the wider marketisation agenda, the character of an undergraduate degree as a good and the structure of the English universities sector have, as I explore in my book, frustrated attempts to make higher education more of a market.

For example, generating product information about a university education is difficult because it is what economists call a credence good – one that is difficult for the consumer to know the quality of, even after it has been consumed.

However, other aspects of the government’s reforms of the previous decade have been more consequential. The removal of student number controls, which capped and distributed places across the sector through state planning, has relocated students away from less popular locations. These institutions – often the ‘squeezed middle’ of the sector, universities with high operating costs and smaller student numbers – are facing a challenging operating environment.

Even though the TEF is now estranged from market making, the effects of competition and choice are still being felt. And it would be naive to assume TEF results will play no role in the attractiveness of some providers who are battling to maintain their market share of students in the coming years.

Andrew Gunn is a lecturer at the University of Manchester, UK.