A ranking alternative worth consideration

What US News and World Report started in the 1980s as an assessment said to be based on objective criteria, has morphed into an international status competition.

The US News’ Best Colleges, Times Higher Education, QS and Shanghai Jiao Tong tertiary education rankings systems attract increasing attention with each autumn.

As with any rankings outcome, there are winners and losers.

Institutions ascending in rank note with muted pride their independently documented quality. After all, if the institution is awarded a higher rank, it must be much better than those of lesser or no rank. Those descending quibble over the validity of the metrics employed, but still draw attention to them.

The media, often the survey sponsors, enjoy additional readership and some seasonal revenue boost with the release of their latest rankings.

Groups that lose out

Two other major interest groups lose out in these annual competitions. The first is obvious – parents and students are by far the largest interest group.

International students and parents considering US institutions face an arduous task with well over 4,000 tertiary institutions in America. That is far too many to critically assess. Yet their decisions carry significant financial burdens and potentially life-changing outcomes.

The major ranking publishers winnow the potential pool down to a more manageable size in ranked order, implying, for instance, that an institution ranked 10th is of higher quality than any institution at a lower position.

Each system employs its own unique mix and weighting of arguably objective input metrics. Their mixes tend to rely on variations in research productivity, peer assessment, selectivity, faculty-student ratios and other input measures.

Their metrics are heavily weighted on input measures with the underlying assumption being that quality inputs unquestionably result in quality output.

I suggest that this heavy reliance on input indicators has a parallel in the kitchen. Starting with the highest quality ingredients assembled by a master chef does not guarantee that the multi-course meal will be edible. How they are blended and prepared at each step of the process will ultimately determine the quality of the result.

The major ranking systems rely heavily, for their yearly publications, on data that are relatively easy to assemble. A degree is a multi-year experience composed of a much more complex mix of ingredients – instructors, pedagogy, courses and a host of other variables.

The second major interest group may be invisible to international parents and students relying on the major rankings.

There are hundreds of otherwise high-quality US institutions lacking the quality patina reflected in the number of research publications, peer reputation, size of endowment or other metrics not necessarily indicative of the quality and value of the total educational experience they provide.

The Washington Monthly ranking

While the major ranking publishers, with the exception of Shanghai Jiao Tong, regularly tweak their mixes and weightings, they do not provide parents and students with indicators of what value-added can be expected in return for the expense of a foreign degree.

Parents and students considering US institutions may want to consider a supplemental or alternative US-based resource, the Washington Monthly, which has published US institutional rankings since 2006.

It employs a mix of familiar input metrics, research output and grants awarded. Its 13 measures are organised into three unique groups: research, service and social mobility.

While many are not materially different from the better-known rankings or may not be of interest to international readers, the Washington Monthly has two output measures that merit special attention.

One is based on the fact that if an institution admits only the brightest and the best applicants, it should not be a surprise that the range between its predicted and actual graduation rates is small.

Without going into the methodology, which is available on its website, institutions with an actual rate exceeding their predicted rate show the relative effectiveness of their instructional programming.

Two, the Washington Monthly presents a complementary institutional cost-adjusted graduation rate. This metric is said to rank institutional efficiency in preparing graduates at lower costs. It is corrected for variances in student demographics and institutional characteristics.

While the ranking includes cost factors that are relevant only to US citizens, international readers might consider it a proxy for the potential return on their possible investment. The Washington Monthly’s measures also bring many institutions that have been largely invisible to the attention of discerning parents and students seeking value instead of the glamour of high input ratings.

Other college ranking systems try to measure which institutions are of the highest quality but not necessarily those that provide a reasonable return on investment. The universities and colleges that dominate these league tables tend to be the wealthiest and most selective institutions with little or no indication of value added.

While parents and students will continue to be drawn to historically highly ranked and priced institutions, including the Washington Monthly’s unique two measures in their discernment are well worth consideration.

Many of the traditionally premier institutions fall short when an alternative quality assessment system is employed.

* William Patrick Leonard is vice dean of SolBridge International School of Business, Daejeon, Republic of Korea.