20 October 2017 Register to receive our free newsletter by email each week
Advanced Search
View Printable Version
AUSTRALIA-SPAIN: A universal intelligence test

'Intelligence quotient', or IQ, tests have long been widely used by psychologists. But what if someone wanted to find out the intelligence of a dolphin, or an elephant, or a computer program and how it compared with a human: what kind of test could assess how smart a machine or a non-human animal is?

At Monash University in Melbourne, Associate Professor David Dowe from the faculty of information technology hopes to devise one that could measure the intelligence of a computer or an animal. Together with Associate Professor José Hernández-Orallo of the Technical University of Valencia and colleagues in Spain, Dowe has set out to develop an "anytime universal intelligence test".

The team of researchers might differ in whether they are likely to achieve their goal in five years or 30, but they are confident of ultimately creating a test that could be applied to a computer or a living thing without using a human language.

But why bother?

Dowe says machines are getting smarter and he refers to the time in 1997 when world chess champion Garry Kasparov was beaten by IBM's Deep Blue computer. Then in February this year, two American quiz champions on a TV show called Jeopardy were beaten by the Watson program, an artificial intelligence computer system.

Then again, computer programs such as the one in Deep Blue or Watson cannot be described as intelligent, Dowe says, because they are programmed to perform quite specific tasks. When a computer starts writing poetry and demonstrating what we would regard as intelligence, however, "then we have to take them more seriously".

"What we are doing is partly intellectual curiosity and partly practical because if machines are becoming smarter we need to know if they can be trusted with more and more responsibility.

"In one paper where José and I describe our work, we refer to the 'technological singularity', the point in the future when machines will overtake us intelligent humans. With a universal test you could do on machines or living creatures, we could check whether we are approaching that point."

The paper, "Measuring Universal Intelligence: Towards an anytime intelligence test", was published last December in the top-rated journal Artificial Intelligence. It became an instant hit and the journal's most downloaded article over the next five months.

Dowe is particularly interested in an aspect of artificial intelligence technology called Minimum Message Length inference, developed at Monash by Professor Chris Wallace in 1968 and now widely applied in machine learning, statistics and data mining. The latter process involves looking for hidden patterns in data that can be used to predict future behaviour - for example, software that can help retail companies find customers with common interests.

He says part of his research concerns the relationship between Minimum Message Length, or 'two-part compression', and the inductive aspect of intelligence involved in learning through recognising patterns.

It is a person's ability to summarise or 'compress' information by detecting patterns that aids problem-solving and this is the key to devising a universal intelligence test that could be applied to machines or non-human animals.

With Hernández and his colleagues Sergio España-Cubillo and Javier Insa-Cabrera from the Technical University, and Associate Professor Maria-Victoria Hernández-Lloreda at the Universidad Complutense in Madrid, Dowe is using computers to generate novel games and patterns to eliminate human bias.

This allows the researchers to generate and tailor tests with any level of complexity they want - even ones beyond the ability of humans to answer.

Constructing such tests and getting results with humans and machines had never been done before, until the team developed and conducted initial trials of a prototype Anytime Universal Intelligence Test.

Different versions of the test were undertaken by 20 staff and students in computer science at the Technical University and by an artificial intelligence program known as Q-Learning. Although Q-Learning is not a sophisticated program, it scored slightly better than the humans.

Dowe says the ambiguity of the initial test results indicates the complexity of moving to a broader understanding of intelligence than the traditional method of using human intellect as the yardstick - a development necessary to determine if, or when, artificial intelligence outstrips humans.

"We are using a mathematically based definition of intelligence, which is based, in simple terms, on the ability to detect patterns of various degrees of complexity. In the future, the test should adapt to the user - becoming more complex if the user is scoring well and more simple if the user is struggling," he says. "

"The question then is how to test the intelligence of an animal such as a dolphin. Dolphins live in a different environment to us but there is research to suggest they call each other by name. It's hard for us to assess how smart they are and clearly intelligence is more than being able to recognise patterns or have a good memory but, while other things are involved, dealing with patterns is certainly a large part of intelligence."

Although the researchers have devised interfaces that can be tested with machines, Dowe says the challenge is to use one that would measure the intelligence of an animal so it could be compared with that of a human.

Techniques have been developed in comparative psychology and ethology where scientists have been evaluating animals for decades, but then the problem arises of presenting the test problem in such a way the animal recognises the pattern.

* For more information about the Anytime Universal Intelligence Test click here.

* This article by Geoff Maslen was first published by The Age newspaper in Melbourne last week. It is published here with permission.
Receive UWN's free weekly e-newsletters

Email address *
First name *
Last name *
Post code / Zip code *
Country *
Organisation / institution *
Job title *
Please send me UWN’s Global Edition      Africa Edition     Both
I receive my email on my mobile phone
I have read the Terms & Conditions *