The web trail – Using cybermetrics to build reputation

In the inaugural lecture of the international conference “Transparency versus Rankings”, held last month at the Polytechnic University of Valencia and led by Francisco Marmolejo, education programme coordinator at the World Bank, two approaches to the future of higher education were presented.

One involved various macro-level factors (especially economic and demographic ones) and the other considered the impact of information and communication technology (ICT) on higher education, both internally (infrastructure and services) and externally (affecting institutional relations and visibility) – issues that are discussed extensively in the current literature.

But one critical aspect not often talked about is that the use of ICT by universities – in addition to changing their structure, how they function or the services they offer – generates a trail that can be quantified and evaluated, which provides complementary information of indisputable value and which, given its size and global sweep, should not be overlooked in any university information system, whether it is able to be evaluated or not.

Cybermetrics is the discipline responsible for the measurement and evaluation of data that relates to the construction, impact and use of information resources, structures and technologies (created, distributed and consumed by different physical or institutional users) on the internet, mainly through quantitative social science methods.

The application of these techniques to studying universities has enabled, among other things, the identification of a digital divide between the US and Europe, the detection of the influence of linguistic and cultural patterns in institutional relationships, and the verification of a growth in academic web visibility in Asia.

University websites

Nevertheless, the main discovery is the idea of a significant link between online and offline activities. The web has demonstrated that it reflects the behaviour of institutions in an accurate way because its usage is so intensive. These studies focus on universities' websites.

Website platforms have become the central point from which institutions build their presence and reputation on the web, providing a reflection of their main activities and missions as well as administrative activities, governance and service management.

University websites have in this way become complex dynamic information systems, due to the diversity of their users and the complexity of their functions, and whose value as a source of information and as a work tool is growing among current and future – especially international – students.

It is therefore vital to know what users want (study programmes, information on fees, admissions criteria, events etc), where they access them from (mobile platforms and social networks) and what they think of the services offered.

Clearly a low quality website harms the perception students have of a university. A recent survey showed that 88% of respondents indicated that Google was easier to search than the university's own website.

What effect does that have on prospective international students, and what university system evaluates this kind of thing?

The use and impact of web resources and services provided by universities can be measured through various web indicators – especially their size, visibility and usage – and have been used to design web ranking of universities, among which the Ranking Web of Universities stands out for its quality and coverage.

These rankings allow the quantification of all university missions as long as these activities are reflected on the university's website, which requires the application of an appropriate web policy.

This comprehensive analysis capability provides cybermetrics with its own characteristics, which other methods do not have and which are essential in an environment of intensive usage of the web – characteristics that are embedded in various university activities such as research (articles and data repositories), teaching, and imminently in knowledge transfer, services and administration, and that are measurable by classical or modern cybermetric indicators, such as altmetrics.

In addition, the social web allows commentary on a global scale on university resources available on the internet, and on activities, projects or institutional decisions.

The quantitative and qualitative analysis of this information is a real challenge to the study of the global reputation of universities and their various units – departments, faculties, research groups etc. The tools and mechanisms needed to perform these analyses are coming from the business world to the academy.

At the university level there are already projects like Klout-Universities or TrackSocial, which generate listings of universities relating to certain parameters associated with universities' institutional accounts on various social networks (which act as university web satellites).

Analysis of the sentiment contained in the comments on these networks will take us a step further. Services like Socialmention, among many others, free and paid for, may reflect, with margins of error, whether the comments about a university (or part of it) are positive, negative or neutral, which raises questions for future evaluation activities, for example:

• • Should a MOOC (massive open online course) taught by a prestigious university with many (or few) students enrolled, and many (or too few) positive (or negative) reviews, be taken into account?
  
• • Should the institutional archive of a university with many (or few) items of dubious quality or provenance, which are rarely viewed or downloaded, be evaluated?

The imminent use of these indicators poses certain challenges for the different actors involved:

• • Universities should develop guidelines to monitor their presence on the web, in terms of both horizontal (different missions) and vertical (different departments) aspects, and sort them according to their strategic aims.
  
• • The publishers of information and evaluation tools must begin to consider the inclusion of certain performance metrics from universities' websites.
  
• • Users should begin to understand and contextualise these indicators properly.

Obviously there are limitations in the use of these indicators: they can be manipulated, there may be inaccuracies in the sources used, and we need data-filtering processes – similar problems exist in bibliometric or economic indicators.

But the social evaluation of universities is a reality that is here to stay. The question therefore is will universities integrate this information into their evaluation and performance mechanisms, or look the other way?

In a globalised and hyper-connected world, hesitation today can mean invisibility tomorrow.

* Enrique Orduña-Malea is a research fellow at the University of Granada.

* Translated from Spanish by Mandy Garner, commentary editor of University World News.