Measuring Artificial Intelligence: the Turing Test

Can Machines Think?

Alan Turing asked himself the same question over six decades ago in his famous article entitled “Computing machinery and intelligence” (1950). His answer was to come up with what is today known as the Turing test, although in his day he called it “The Imitation Game” – a name that’s now familiar to everyone, thanks to the movie based on one of the most interesting episodes in the life of the mathematical genius: the time he spent deciphering the Enigma code, which played such a key role in the outcome of the Second World War.

But as well as the end of the war (in part), we also owe many other things to Turing. Considered as one of the fathers of information technology, in the last century he was already pondering the principle of one of the great questions of today’s society: in what direction are machines evolving, and what form will their interactions with human beings take?

Although we’re still very far from the scenario of technological apocalypse suggested by a fair number of movies and books, evidence from new developments in ICT now makes this question much easier to answer. Sixty years ago Turing could not have foreseen the current situation, and yet he still decided to design a method that would respond scientifically to the question of whether a machine could think for itself or not. He created the Turing test, a conversation in natural language between a human being and a machine designed to generate a verbal interaction in which it is impossible to tell the difference between human and software. Five minutes of conversation to convince (at least 30% of the time) the person assessing the chat that whoever is behind the screen (who only expresses him/her/itself through text, like in a virtual chat room) is a human being. If it can do so, the machine will have passed the test.

The first to “pass” the Turing test

This took place in 2014, over six decades after the tragic death of the British mathematician on 7 June 1954. Two years have already passed since the controversial experiment performed by a controversial scientist, Kevin Warwick. The results –as might be expected– were also controversial, and rekindled the debate on artificial intelligence. Warwick has spent his whole life studying artificial intelligence, and is correctly experimenting with robotics and the world of cyborgs at Reading University. Chips, electrostimulators and a whole array of issues to do with the potential of the human brain, in combination with technological tools, are all part of this researcher’s everyday work. In 2014 he organized the largest-scale Turing test in history: 30 judges and five machines who took part in a total of 300 conversations.

The first to pass the Turing test was Eugene Goostman, a chatbot developed by its programmers to simulate the personality of a Ukrainian adolescent. This aspect played in his favor during the test: as he didn’t imitate the conversation of an adult, he was able to naturally mimic the lack of knowledge characteristic of his age. Eugene only just passed, with 33%, and his achievement raised many questions and objections within the scientific community. “They were the official parameters established by Turing”, said Kevin Warwick, the official organizer, in the British newspaper “The Independent“ a few days after the event.

At the time, Turing laid down a series of conditions, such as avoiding mathematical questions, although he never mentioned anything about not including “children”. The controversial pass mark in 2014 opened up the debate between those who consider the Turing test to be the cornerstone of AI (Warwick himself defined it as such) and those who doubt this method can answer the question of whether a machine “thinks” for itself or not.

However it is all a question of perspective, even in the world of artificial intelligence. Eugene’s result also proves that in 66.7% of cases machines do not succeed in taking in the judges, and so we can rule out (for the moment) a successful software rebellion. But it begs the question of whether this test also assesses “natural intelligence” –the typical intelligence of human beings. What happens if the results say you’re a machine?

Dory Gascueña for OpenMind

@dorygascu

Source: OpenMind