The academic world urgently needs a critical discussion on what is meant by 'good' research. While there is no single yardstick for assessing research quality across all disciplines, regions and cultures, this does not make the question of quality irrelevant. On the contrary, the issue concerning appropriate criteria for assessing the quality of research should be on the agenda wherever it is being conducted.
Writing in the Research Report of the UNESCO Forum on Higher Education, Research and Knowledge, authors Urich Teichler and Yasemin Yagci note that while the elements all serious research activities have in common do exist, it is hard to conceive of 'good' research without decent evidence and without an explicit, transparent set of methodological ground rules.
"Beyond that, however, different purposes, different kinds, different traditions of research do need to examine their own criteria," they write in the chapter titled "Changing Challenges of Academic Work: Concepts and observations".
"The process of discussing these reflections on criteria across the international world of research should be one of the most exciting chapters of future cooperation."
Efforts to formulate the basis of excellent teaching and research have sounded similar around the world over the last few decades, Teichler and Yagci say. They quote one commentator who identified the characteristics that all research universities share as including high-quality academics committed to research and teaching, high-quality graduate students who want to learn to perform research, an intellectual climate that encourages scholarship, adequate research funding and appropriate research infrastructure.
But they also point to the contrasts between the 'massification' of higher education in most countries and the belief that the quality of teaching and research is becoming endangered. Similarly, although more and more students are prepared for and motivated by quality education, governments seem less willing to maintain average expenditures per student at a high level.
Then there is the 'privatisation' of higher education where private provision is claimed to be closely linked to the economy and the needs of employers. This leads to a concentration on fields in high demand, a curriculum geared to the workplace, more teaching by practitioners and more practical experience during courses. Yet the authors refer to experts who suggest study provisions at many private institutions are of poor quality, in some cases so low that graduates have difficulty gaining employment.
Another issue is the loss of autonomy that universities and academics enjoyed for many years in the wealthier economies, where they were free to a large extent of government intervention. In recent decades, however, governments have intruded on autonomy through "increased incentive steering, ways of boosting competition, the establishment and extension of evaluation schemes, strengthening managerial power within higher education and research institutions and ensuring greater stakeholder influence," write Teichler and Yagci.
"Advocates of this move from an environment of trust [in universities to manage their own affairs] towards a mix of operational steering instruments to increase effectiveness and efficiency, tend to claim these mechanisms enhance the quality of higher education and research, while leaving open what the consequences might be for the relationship between quality and relevance," the authors continue.
They note that in most low to middle-income countries, the relationships between governments, university managements and academics, as well as funding conditions, are less based on trust.
Research funding is one of the key areas where a substantial move from 'trust' to incentive and competition has taken place - even though the move has not been uniform and considerable differences exist between countries regarding the portion of funds based on incentives, the criteria used and their overall effects.
"In an increasing number of research activities, however, researchers are expected to ensure that quality and relevance will play a major role in their activities. British expert John Brennan argues that 'pressures for greater accountability and performativity bring with them new types of requirements for relevance and in particular the need to find measures of it... if you want to obtain a research grant it is probably more important today than in previous periods to make a serious claim for the potential societal relevance of the proposed research'."
In many low and middle-income countries, the establishment of a national research promotion system providing funds for individual research projects on the basis of reviewing and assessing research proposals is viewed as a breakthrough towards quality-driven research funding. The descriptions of these systems often point to the transparency and fairness of the research promotion system more than the weight of quality and relevance criteria, Teichler and Yagci say.
They argue that many experts say research in low and middle-income countries is driven, and must be more strongly driven, by imperatives of relevance. But this does not mean that debates about desirable and acceptable criteria of relevance are less heated and less controversial, rather that "researchers from developing countries often note they are expected, by research epistemologically driven by the North, to integrate prevailing themes and paradigms in order to become part of the world-wide accepted community of research 'quality'."
Teichler and Yagci note that tensions between research quality and relevance in the developing world were summarised in sections of the report, Universities as Centres of Research and Knowledge Creation: An endangered species? published by the UNESCO Forum last year.
In a section titled "Rethinking the criteria for research quality", that report says substantial criticism surrounds the notion of a one-dimensional set of criteria for assessing the quality of research regardless of where, by whom, and on what subject it was performed. Instead, dealing with assessment of research quality requires more differentiated ways, taking into account the research setting, the kinds of research questions asked, the methodological orientation, and the use of research findings.
Another section, "Different functions and criteria of research" notes that while all societies need knowledge, differences exist in the functions that university research can serve. Universities such as Stanford, much more generously endowed and equipped than most, may be able to serve a broader range of functions in a global context, even beyond the needs of their particular environment and society.
Others, such as the University of Ghana, have to concentrate on the present and future knowledge needs of their own communities: "Relevance and utility of research have to be seen and judged with these distinctions in mind, and are not amenable to one-dimensional rankings."
This leads to the next section, "Rankings, globalisation and relevance", where the report refers to the fascination all universities have with league tables and rankings as being "as understandable as it is problematic" because certain universities, by virtue of their location, are much more likely to appear in these tables than others.
The question arises then of how universities reconcile the often conflicting demands of international competitiveness while meeting their obligations to the knowledge needs of their local communities. In cases of irreconcilable conflict, which should take precedence? The question is important not only for universities but also for agencies that fund university research, which should recognise this conflict and the need for universities to balance these competing expectations.
"Faced with this dilemma, it is worth considering much more seriously whether there should not be alternative kinds of ranking that take more explicitly into account the degree to which universities and their research programmes serve the knowledge needs of their local communities and societies, without necessarily compromising the standards of what is internationally considered good research," Teichler and Yagci conclude.
* Professor Ulrich Teichler is immediate past Director of the International Centre for Higher Education Research (INCHER-Kassel) at the University of Kassel in Germany. He has published more than 1,000 academic publications, especially on higher education and the world of work, international comparisons of higher education systems, and international cooperation and mobility in higher education.
* Yasemin Yağci is a PhD student and research assistant in the International Centre for Higher Education Research (INCHER-Kassel) at the University of Kassel in Germany.
Click here to download the Unesco Forum Research Report.
Receive UWN's free weekly e-newsletters