GLOBAL: The challenge of quality assurance

How the quality of a university education can be assured and standards of teaching and research maintained is an issue that governments and higher education institutions around the world continue to confront. In a paper discussing how different countries have tackled this topic, as Australia is at present, the Executive Director of the Australian Group of Eight universities, MICHAEL GALLAGHER, considers the responsibility of government and the university in assuring quality.

Governments are responsible for responding in a timely, coherent and cost-effective manner to matters of concern to their communities. In broad terms, the community looks to centralgovernments primarily to tackle the big picture issues rather than become involved in micro-level matters that can be devolved to other levels of government or undertaken by specialist service providers whether public or private.

In higher education, the roles of central government include providing policy direction, incentives and regulatory frameworks for structuring the supply of education services to accommodate student demand and producing an educated citizenry and workforce to meet community needs. Particular initiatives may include adequate funding for institutions, the provision of financial assistance to students, and incentives to promote equity of access and ease transitions across education, training and work.

Understandably, governments have an interest in ensuring that the goals of universities and other institutions align with their wider social and economic goals. However, they normally 'steer from a distance' rather than engage regularly in direct monitoring of institutional effectiveness, which is properly the responsibility of university governing bodies.

It is proper not only because it reflects an appropriate separation of accountabilities but also because it is most practical, given that central government policymakers and administrators have little capacity to review and act on the kinds of qualitative outcomes assessment data that are collected and reviewed on an ongoing basis within universities. It is best that governments focus on the areas where they can best add value and be least distracted:

"An accountability system focused on state-level concerns is designed to answer questions to inform state policy decisions about system design, governance, articulation, and finance. It asks how are we, collectively as a state, doing in achieving our goals. An accountability system designed principally to collect and review data on institutional performance asks a totally different question: how well is an individual college or university accomplishing its unique mission?

"This institutional focus has several problems: it diverts state policy-makers from the issues that they can influence through their responsibility to make public policy; it leads to micromanagement over institutions, whose own governing boards are responsible to monitor and manage institutional effectiveness; and it overloads state accountability systems with far more data than users can possibly digest and use.

"Finally, by reflecting a top-down 'we (policy-makers) are holding you (college) accountable for your performance' approach, it invites arguments about the adequacy of funding, factors outside the college's control, and the overall fairness of the top-down assessment" (Shulock, 2003).

Universities as self-governing institutions are responsible for achieving the goals they set consistent with their missions. They respond to policy, administrative and market signals, and they account to the communities that support them for the stewardship of their resources. They have particular responsibilities regarding the selection and admission of students, curriculum, pedagogy, assessment, and the provision of student services and alumni services, alongside the professional development of academic and administrative staff.

They also have responsibilities for the ethical conduct of research, the preservation of knowledge and the publication of scholarly outputs, and contributions to national, regional and community development. And they have responsibilities for continuous improvement in all that they do. With regard to the accountabilities of universities to the students they admit, Scott (2008) has suggested four foci: design; support; delivery; and effect.

The first of these (design) requires a university to make clear what it stands for (mission and values), what it seeks to achieve (fitness of purpose) and how it structures the learning experiences it offers (fitness for purpose). The second (support) and third (delivery) areas are to do with how well the designed approach is implemented. The fourth dimension (impact) is where the rubber hits the road: how well do students learn, and how useful is their learning to their life prospects?

It is reasonable to expect that each university will have in place processes to validate its mission, review the fitness of its programme design, verify that its delivery meets the standards it has set, and know how well its students and graduates perform. These processes as well as the results they achieve should normally be subject to systematic internal review and independent external review.

Governments have two main policy options in the design of their relations with universities: a principal agent model of accountability, or a mutual responsibility model of shared development. The latter is the more likely to lead to performance improvement, and to do so in ways that enable broader service delivery effectiveness. Accountability involves rendering an account about what one is doing in relation to the goals and expectations of others

Accountability obligations are established when an agent accepts resources and responsibilities from a principal. The principal/agent model of contractual relationships prioritises the agent's compliance with the purposes and performance and information requirements of the principal over their wider stewardship responsibilities for serving broader public needs. The emphasis on reporting the measurable aspects of performance can reduce service delivery effectiveness.

The principal/agent model of accountability contrasts with the mutual accountability model, wherein goals are shared and there is 'buy-in' to responsibilities through "developing shared understanding, respect, trust and mutual influence". A principal-agent approach would work from the premise of an expectation that a university ought to comply with standards set by others. In contrast, a mutual responsibility approach would start from an acceptance by others of a university's self-set standards.

Only if a review of the self-set standards (fitness for purpose) were to find deficiencies would it be reasonable for government to require adherence to an externally determined set of standards. Within a mutual responsibility agreement, in relation to the agenda compelling governments, the following expectations of higher education institutions may be proposed:

* The probity threshold: Higher education institutions should be able to demonstrate that they have adequate capacity and integrity to deliver what they undertake to deliver.
* The effectiveness imperative: Higher education institutions should have verifiable means for knowing how well students are acquiring the knowledge, understandings and abilities expected.
* The transparency requirement: Higher education institutions should publish information about their distinctive offerings and requirements, along with clear criteria and codified procedures for judgements relating to admission, credit, and assessment grades.
* The comparability challenge: Higher education institutions should make available reliable information to enable students and employers to see similarities and dissimilarities between different programmes, learning opportunities, expected standards, and graduate attainment and outcomes.

For the accountability and improvement purposes of higher education quality assurance to be reconciled in the contemporary circumstances of post-mass participation and diversity of provision, it may be useful also to distinguish between 'normative' or 'common accountability' and 'customised accountability'.

Normative accountability seeks to ensure that higher education institutions conform to a set of basic requirements. It involves the use of common benchmarks for comparing the capacity and performance of institutions. The common benchmarks may include prescribed minimum standards in respect of inputs, processes or outputs, and they may include prescribed measures and instruments for reporting.

Customised accountability seeks to ensure that a higher education institution delivers on its promises. It involves the use of selective indicators designed to measure how well each institution performs relative to its own goals and expectations:

"Locally developed measures have the potential to represent accurately the specific institutional outcomes of higher education given their proximity to what is assessed, rather than standardised assessments, which are distant from the missions and objectives of individual institutions" (Allen and Bresciani, 2003).

It should be possible to agree on a framework that provides for both common and customised accountability and transparency. In discussing the application of competence frameworks, Grant has noted the coexistence of compelling cases for both common and specific frameworks (Grant, 2006).

On the one hand, different companies have specific sets of competences, along with generic skills, for performing the activities of their business, and educational institutions may be motivated to emphasise the particular competences that distinguish their graduates from those of other institutions.

On the other hand, labour mobility requires that individuals educated or trained in one place should be able to find work elsewhere, and employers and professional bodies need to know that graduates meet required standards of competence. Grant suggests the need for a judicious blend of common and specific approaches:

"An insistence on a completely common framework would deny the freedom to experiment, and the freedom for views to differ about which competences are necessary for which roles. But a fragmented approach, where every organisation has its own competence framework, would make life very difficult for self-directed lifelong learners with multiple, diverse and complex career paths" (Grant, 2006).

Grant suggests a meta-framework that allows for two interrelated kinds of competence frameworks: one that is relatively loose and amenable to agreement among different stakeholders, for generic shared competency definitions in particular domains; and the other that is more specifically designed to suit the requirements of a particular body.

Thus, in respect of universities, the government's standards-based agenda could constructively accommodate a judicious blend of (a) a menu of common indicators from which universities select those that are most appropriate to their circumstances, and (b) a set of customised indicators that reflect the objectives and standards that each university has determined.

The questions subsequently to be addressed are (a) which common standards, in terms of their (i) range and (ii) composition, (b) the balance between common and customised standards, (c) the balance between whole-of-institution and programme-specific standards, and (d) the means by which performance standards are verified.

These are significant questions. They are matters for judgement rather than technical considerations. They are so fundamentally important that they need to be referred to public consultation and dealt with openly and systematically, not settled in-house or merely by conference with a limited group or by otherwise cosy agreements.

Given the recent opacity of considerations about these matters [in Australia], any attempt to arrive at apparent consensus through closed dealings will lack legitimacy.

* This is an extract from The Accountability for Quality Agenda in Higher Education, a paper prepared by Michael Gallagher for Australia's Group of Eight research-intensive universities.

References

Allen, J. and Bresciani, M. (2003) "On the Transparency of Assessment Results". Change. January/February.

Grant, S. (2006) Framework of competence: common or specific? In Koper, R. and Stefanov, K. Learning Networks for Lifelong Competence Development Proceedings, Sofia Bulgaria.

Scott, G. (2008) "University student engagement and satisfaction with learning and teaching". Prepared in response to Request for Research and Analysis, Review of Australian Higher Education. Canberra.

Shulock, N. (2003) "A Fundamentally New Approach to Accountability: Putting state policy issues first." Paper prepared for 28th Annual Conference, Association for the Study of Higher Education Forum on Public Policy and Higher Education. Portland, Oregon. Institute for Higher Education Leadership and Policy. Califiornia State University. Sacramento.