The Turing test is an empirical test, the idea of which was proposed by Alan Turing in the article "Computing Machinery and Intelligence", published in 1950 in the philosophical journal Mind. Turing set out to determine whether a machine could think.
The standard interpretation of this test is as follows:
“A person interacts with one computer and one person. Based on the answers to the questions, he must determine who he is talking to: a person or a computer program. The purpose of a computer program is to mislead a person into making the wrong choice.”
All test participants cannot see each other. If the judge cannot say for sure which of the interlocutors is human, then the machine is considered to have passed the test. To test the intelligence of the machine, and not its ability to recognize spoken language, the conversation is conducted in “text only” mode, for example, using a keyboard and a screen (an intermediary computer). Correspondence should occur at controlled intervals so that the judge cannot draw conclusions based on the speed of responses. In Turing's time, computers were slower than humans. Now this rule is necessary because they react much faster than humans.
As of 2009, no existing computer system came close to passing the test.
Computer program simulating a psychotherapist
From Roger Penrose's book The King's New Mind
...The most consistent with the Turing test is a computer program developed by C. M. Colby in the mid-1960s, which imitates a psychotherapist - so successfully that some patients have shown a willingness to prefer it to a human therapist and reveal more about themselves than when communicating with a human therapist. living doctor. As an example of such communication between a computer “psychotherapist” and a patient, we give the following excerpt from the dialogue (which was conducted via teletype):
Therapist: How can I help you?
Patient: I don't know...
Therapist: Try to say something about it.
Patient: No.
Therapist: May I know the reason?
Patient: I'm tired today and don't feel well.
Therapist: Why? Do you have any assumptions about this?
Patient: Yes... I'm old.
Therapist: Please continue.
Patient: It seems to me that I have lost interest in everything that is happening.
Therapist: Tell me more about your feelings.
Patient: I have some thoughts about this...
Therapist: You have some ideas.
Patient: I think I miss my family.
Therapist: What makes you worry about missing your family?
Patient: There is some kind of challenge in this...
Therapist: Why is this so important to you?
Although this dialogue may give rise to the strange sensation of the computer understanding the patient's problem, there is really nothing of the sort: the computer is simply following fairly simple mechanical rules. (There is also the “reverse” situation, implemented in a system where a computer simulating a person suffering from schizophrenia gives textbook answers and simulates relevant symptoms that can fool some medical students into believing that they are talking to a real live patient! )
Robots endowed with consciousness, capable of thinking, creating, communicating - we all know about this idea firsthand, because every now and then new films appear on the screens in which cyborgs appear, for example, “The Terminator”, “I am Robot”, “ Prometheus”, etc. And the news is increasingly full of headlines that scientists have made progress in creating artificial intelligence, have created a humanoid machine, or that the Turing test has finally been passed. And those who are interested in the idea of artificial intelligence, as well as those who do not know what kind of Turing test this is and why there is so much hype around it, will certainly be interested in the material that we have prepared.
Let's start with a brief overview of the topic of artificial intelligence.
Artificial intelligence
The term “artificial intelligence” is commonly understood as the science and technology of creating intelligent machines and, most of all, intelligent computer programs. However, this concept is often also used to refer to the ability of various intelligent systems to perform, which were initially considered to be characteristic only of humans. In addition, the idea of artificial intelligence is associated with a similar task of using computer technology to understand human intelligence.
As a scientific direction, artificial intelligence began to take shape around the middle of the last century, when a sufficient number of prerequisites for this were formed. For example, philosophers increasingly debated the topic of human nature and the processes of cognition of the world, psychologists and neurophysiologists were developing theories concerning thinking and the functioning of the human brain, mathematicians and economists carried out all kinds of calculations and representations of knowledge about the world, etc. Thus, the foundation of the theory of algorithms was laid, thanks to which the first computers appeared.
Practice has shown that the potential of machines in terms of performing calculations is much greater than that of humans, which is why the question arose among scientists: what is the general potential of computers, and will machines eventually be able to reach the level of human development?
So, in 1950, one of the founders in the field of computer technology, the English scientist Alan Turing, wrote an article in the publication “Mind”, “Computing Machines and the Mind,” which described a procedure through which it is possible to determine the moment when the machine becomes one step higher. with a person in terms of his rationality. This procedure is called the Turing test. But, it should be assumed that the implementation of the Turing test would have been impossible without the created artificial intelligence, and scientists approached this process from several positions.
Approaches to creating artificial intelligence
First of all, we should highlight the general approach to creating artificial intelligence, which assumes that it will be capable of displaying behavior that will not differ from human behavior. By the way, the presented idea can be called a generalization of the approach of the Turing test, according to which a machine can be considered intelligent when it is able to conduct a conversation with an ordinary person, and he will not be able to distinguish it from another person, provided that the conversation is in written form.
The second approach belongs to science fiction writers. He says that artificial intelligence will appear when a machine can create and feel. But this approach does not stand up to criticism if we consider it in more detail. For example, creating a machine that will evaluate any parameters of the internal or external environment and somehow react to them, by and large, is not difficult. Considering that the sensors will react to certain stimuli, we can already say about such a mechanism that it has “feelings”.
The third campaign is called symbolic. In fact, it was precisely this that was originally characteristic of the era of digital machines. After the first symbolic computing language was created, developers became confident that there was a chance for the practical implementation of artificial intelligence using symbolic computing technology, which would make it possible to deal with weakly formalized meanings and representations.
There was also a logical approach to the creation of artificial intelligence, based on modeling reasoning, and the main trump card of which was. But since the 90s of the last century, an agent-based approach began to develop, based on the use of intelligent agents and suggesting that intelligence is specifically the computational component of a machine’s potential to achieve its goals.
As a result, a hybrid approach emerged, the main idea of which is that only the integrated use of symbolic and neural models will help achieve a full range of computational and cognitive capabilities. For example, neural networks can generate expert reasoning rules, and through statistical learning generative rules can be generated.
Now let's return to the Alan Turing test.
Turing test
The Turing test, as we mentioned, is designed to determine the potential of artificial intelligence close to. The classic interpretation of this test can be expressed as follows: a person interacts with one computer or person. Based on the answers to certain questions, a person must determine who his interlocutor is: a computer or a person. And the functions of a computer program include misleading a person and leading him to the wrong conclusion. During the test, none of the participants sees each other.
Alan Turing said that a machine can be considered thinking if it can deceive 30% of human participants in the process of correspondence. And for decades, no one was able to implement this test, but in 2014, The Independent reported that a computer program was able to convince people that they were exchanging messages with a thirteen-year-old boy. This, in fact, means that in fact the Turing test was passed.
It is interesting that the computer program called “Eugene Gustman” was created by Russian programmers, and the test itself was organized by scientists from the University of Reading in London at the Royal Society. During the experiment, 33% of participants believed that they were communicating with a real person. By the way, “Eugene” claimed that he was a thirteen-year-old boy living in Odessa.
This fact is undoubtedly a huge step in artificial intelligence research, but scientists say that this achievement could be used by cybercriminals.
Eugene Gustman is the first program to pass the test, although there are other programs that come close. Among them are “JFRED”, “Elbot the Robot”, “Ultra Hal” and “Cleverbot”.
According to one of the founders of Eugene, Vladimir Veselov, the main idea of the developers was that Eugene knew everything and, at the same time, knew nothing, and a huge amount of time was spent on developing a program with a “real” personality. University of Reading professor Kevin Warwick says there has never been a more controversial and significant event in the field of artificial intelligence.
In this case, we cannot but agree with the experts, because the very fact that a machine was able to convince people that it is a person can turn the entire world computer system upside down. Just imagine what the consequences could be if cybernetic interlocutors with artificial intelligence are able to conduct live correspondence with people, and spam bots become impossible to recognize?
Conclusion
What can be said about the idea of artificial intelligence? On the one hand, it is truly amazing, and if artificial intelligence is created, it will allow all of humanity to take a huge step forward in its development. But if you look at it from a critical point of view, a conscious artificial intelligence, if it falls into the hands of unscrupulous people, can cause harm beyond description to a person. You can philosophize on this topic for a very, very long time, but we will not engage in this - let it become food for your mind.
We only want to advise you to study and become smarter and more educated, because no machine will do this for you.
Text
Artyom Luchko
Britain's University of Reading announced with great fanfare that a "major milestone in the history of computing" had been passed and a computer had passed the Turing test correctly for the first time, misleading judges into believing it was communicating with a 13-year-old Ukrainian boy. Look At Me figured out what really lies behind this event.
What was the experiment
The University of Reading, which conducted the first successful Turing test
The chatbot trial was organized by the School of Systems Engineering at the University of Reading to mark the 60th anniversary of Alan Turing's death. The experts communicated simultaneously with a live person and with the program, being in different rooms. At the end of the test, each judge must declare which of his two interlocutors is a person and which is a program. For the purity of the experiment, five computers and 30 judges were used, each of whom conducted a series of 10 written dialogues lasting 5 minutes. Although usually in the annual competition of artificial intelligence programs for the Loebner Prize ( in which programs compete to pass the Turing test for a prize of $2000) Only 4 chatbots and 4 people take part. As a result of the experiment, the Eugene Goostman program managed to convince 33% of the jury of its “humanity,” which happened for the first time in history. Robert Llewellyn, one of the judges, a British actor and technology enthusiast, said:
The Turing Test was amazing. There were 10 sessions of 5 minutes each, 2 screens, 1 person and 1 machine. I guessed correctly only 4 times. This robot turned out to be a smart guy...
Chatbot Eugene Goostman was developed by Russian native Vladimir Veselov (he now lives in the USA) and Ukrainian Evgeniy Demchenko, living in Russia. The first version appeared back in 2001. The teenager’s age was not chosen by chance: at the age of 13, a child already knows a lot, but not everything, which complicates the judges’ task. In 2012, the chatbot had already come quite close to success: then 29% of the judges believed in the “humanity” of the Ukrainian schoolchild. During the latest improvements, programmers were able to prepare the virtual interlocutor for all possible questions and even teach him to select example answers on Twitter.
What is the Turing Test,
and what are its disadvantages
Alan Turing aged 16
The Turing test was first proposed by British mathematician Alan Turing in his paper “Computing and Intelligence,” published in the journal Mind in 1950. In it, the scientist asked a simple question: “Can a machine think.” In its simplest form, the test is as follows: a person interacts with one computer and one person. Based on the answers to the questions, he must determine who he is talking to: a person or a computer program. The purpose of a computer program is to mislead a person into making the wrong choice. The test involves a five-minute text conversation, during which at least 30% of the judges must believe that they are dealing with a person and not a machine. In this case, of course, all test participants do not see each other.
John Searle, American philosopher
There are many different versions of this test (in some variations the judge knows that one of the interlocutors being tested is a computer, in others he does not know about it), but many scientists and philosophers criticize him to this day. American philosopher John Searle challenged the test with a thought experiment known as the “Chinese Room.” He allowed himself to suggest that the ability of a computer to carry on a conversation and answer questions convincingly is far from the same as having a mind and thinking like a person. “Suppose I were locked in a room and [...] that I did not know a single word of Chinese, either written or spoken,” Searle writes in 1980. He imagined that he was receiving questions written in Chinese through a crack in the wall. He was not able to read these symbols, but had a set of instructions in English that allowed him to respond to "one set of formal symbols with another set of formal symbols." Thus, Searle would theoretically be able to answer questions simply by following the rules of English and choosing the correct Chinese characters. And his interlocutors would be convinced that he could speak Chinese.
Most critics of the Turing test as a way to evaluate artificial intelligence are of a similar opinion. They argue that computers can only use sets of rules and huge databases programmed to answer questions to appear intelligent.
How the program deceived the jury
Reading University Professor Kevin Warwick
Eugene Goostman had two factors that helped him pass the test. Firstly, grammatical and stylistic errors that the machine makes in imitation of a teenager’s writing, and secondly, a lack of knowledge about specific cultural and historical facts, which can also be attributed to the age of the student.
There is no stage in the development of artificial intelligence more iconic or controversial than passing the Turing test.
"The success of the program is likely to raise some concerns about the future of information technology," said University of Reading professor Kevin Warwick. - There is no more iconic or controversial stage in the development of artificial intelligence than passing the Turing test, when a computer convinces enough judges to believe that it is not a machine, but a person, communicating with them. The very existence of a computer that can trick a person into thinking it is a human is a red flag for cybercrime.” The Turing Test is still an important tool in combating this threat. And now experts have to more fully understand how the emergence of such advanced chatbots can affect online communication on the Internet.
Judging by the logs that can be found on the Internet (it’s not yet possible to try the bot on your own; probably, due to the hype, the site couldn’t handle the traffic and “fell”), The chatbot is quite primitive and, as it seems at first glance, is not very different from similar developments that can be found on the Internet. One of the interesting dialogues with “Eugene” was presented by journalist Leonid Bershidsky, who asked him uncomfortable questions about a high-profile event that could not pass by the young Odessa resident.
Even taking into account the well-developed character and biography, mistakes and typos that a real teenager can make, the persuasiveness of the bot is questionable. In fact, it also reacts to keywords, and when it is stumped, it produces pre-prepared and not the most original placeholder answers. If the program had the ability to use search engines to place itself in the context of the current world situation, we could see a much more impressive result. This will probably take time. Previously, the famous futurist Raymond Kurzweil, who holds the position of technical director at Google, stated that computers will be able to easily pass the Turing test by 2029. According to his assumptions, by this time they will be able to master the human language and surpass humans in intelligence.
7 supercomputers that can outsmart humans
ELIZA
An empirical experiment in which a person communicates with a computer intelligent program that simulates responses like a person.
It is assumed that Turing test passed if a person, when communicating with a machine, believes that he is communicating with a person and not a machine.
British mathematician Alan Turing in 1950 came up with such an experiment by analogy with an imitation game, which assumes that 2 people go into different rooms, and the 3rd person must understand who is where by communicating with them in writing.
Turing proposed playing such a game with a machine, and if the machine could deceive an expert, this would mean that the machine could think. Thus, the classic test follows the following scenario:
A human expert communicates via chat with a chatbot and other people. At the end of the conversation, the expert must understand which of the interlocutors was human and which was a bot.
Nowadays, the Turing test has received many different modifications, let's consider some of them:
Reverse Turing test
The test consists of performing some actions to confirm that you are a person. For example, we may often be faced with the need to enter numbers and letters into a special field from a distorted image with a set of numbers and letters. These actions protect the site from bots. Passing this test would confirm the machine's ability to perceive complex distorted images, but such do not exist yet.
Immortality test
The test consists of repeating a person’s personal characteristics as much as possible. It is believed that if a person’s character is copied as accurately as possible and cannot be distinguished from the source, it means that the test of immortality has been passed.
Minimal intelligent Signal test
The test assumes a simplified form of answering questions - only yes and no.
Meta Turing Test
The test assumes that a machine “can think” if it can create something that it itself wants to test for intelligence.
The first passage of the classical Turing test was recorded on June 6, 2014 by the chatbot “Zhenya Gustman”, developed in St. Petersburg. The bot convinced experts that they were communicating with a 13-year-old teenager from Odessa.
In general, the machines are already capable of a lot, now many specialists are working in this direction and more and more interesting variations and passing this test await us.
"Eugene Goostman" managed to pass the Turing test and convince 33% of judges that it was not a machine communicating with them. The program posed as a thirteen-year-old boy named Evgeny Gustman from Odessa and was able to convince the people talking to it that the answers it produced belonged to a person.
The test took place at the Royal Society of London and was organized by the University of Reading, UK. The authors of the program are Russian engineer Vladimir Veselov, who currently lives in the United States, and Ukrainian Evgeniy Demchenko, who now lives in Russia.
How did the program "Evgeniy Gustman" pass the Turing test?
On Saturday, June 7, 2014, a supercomputer named Eugene tried to recreate the intelligence of a thirteen-year-old teenager, Evgeny Gustman.
Five supercomputers participated in the testing, organized by the School of Systems Engineering at the University of Reading (UK). The test consisted of a series of five-minute written dialogues.
The program developers managed to prepare the bot for all possible questions and even train it to collect examples of dialogues via Twitter. In addition, the engineers endowed the hero with a bright character. Pretending to be a 13-year-old boy, the virtual “Evgeny Gustman” did not raise doubts among experts. They believed that the boy might not know the answers to many questions, because the average child’s level of knowledge is significantly lower than that of adults. At the same time, his correct and accurate answers were attributed to unusual erudition and erudition.
The test involved 25 “hidden” people and 5 chatbots. Each of the 30 judges conducted five chat sessions, trying to determine the real nature of the interlocutor. For comparison, in the traditional annual competition for artificial intelligence programs for the Loebner Prize*, only 4 programs and 4 hidden people participate.
The first program with a “young Odessa resident” appeared back in 2001. However, only in 2012 did she show a truly serious result, convincing 29% of the judges.
This fact proves that in the near future, programs will appear that will be able to pass without problems Turing test.
So today we will talk about the most famous test for evaluating a talking bot - the Turing test.
The Turing test is an empirical test, the idea of which was proposed by Alan Turing in the article "Computing Machinery and Intelligence", published in 1950 in the philosophical journal Mind. Turing set out to determine whether a machine could think.
The standard sound of the law: “If a computer can operate in such a way that a person is unable to determine whether he is communicating with another person or with a machine, he is considered to have passed the Turing test.”
Intelligent, human-like machines have been a major theme in science fiction for many decades. Since the birth of modern computing technology, people's minds have been occupied by the question: is it possible to build a machine that could in some way replace a person. An attempt to create solid empirical ground for resolving this issue was the test developed by Alan Turing.
The first version of the test, published in 1950, was somewhat confusing. The modern version of the Turing test is the following task. A group of experts communicates with an unknown creature. They do not see their interlocutor and can communicate with him only through some kind of isolating system - for example, a keyboard. They are allowed to ask their interlocutor any questions and conduct a conversation on any topic. If at the end of the experiment they cannot tell whether they were talking to a person or a machine, and if in fact they were talking to a machine, the machine can be considered to have passed the Turing test. 
There are at least three main versions of the Turing test, two of which were proposed in the article "Computing Machines and Intelligence", and the third version, in Saul Traiger's terminology, is the standard interpretation.
While there is some debate as to whether the modern interpretation corresponds to what Turing described or is the result of a misinterpretation of his work, the three versions are not considered equivalent, and their strengths and weaknesses differ.
Imitation game
Turing, as we already know, described a simple party game that involves a minimum of three players. Player A is a man, Player B is a woman and Player C, who plays as the conversation leader, is of any gender. According to the rules of the game, C does not see either A or B and can communicate with them only through written messages. By asking questions to players A and B, C tries to determine which of them is a man and which is a woman. Player A's job is to confuse player C so that he makes the wrong conclusion. At the same time, player B's task is to help player C make the right judgment.
In what S. G. Sterret calls the Original Imitation Game Test, Turing proposes that the role of Player A be played by a computer. Thus, the computer's task is to pretend to be a woman in order to confuse player C. The success of such a task is assessed by comparing the outcomes of the game when player A is a computer and the outcomes when player A is a man. If, in Turing's words, "a conversational player makes the wrong decision as often after a game [with a computer] as after a game between a man and a woman," then the computer can be said to be intelligent.
The second option was proposed by Turing in the same article. As in the Initial Test, the role of Player A is played by the computer. The difference is that the role of Player B can be played by either a man or a woman.
“Let's look at a specific computer. Is it true that by modifying this computer to have sufficient storage space, increasing its speed, and giving it a suitable program, it is possible to construct such a computer so that it satisfactorily plays the role of player A in a simulation game, while the role of player B is performed by a man?" - Turing, 1950, p. 442.
In this variation, both players A and B try to persuade the leader to make an incorrect decision.
The main idea of this version is that the purpose of the Turing test is not to answer the question of whether a machine can fool a leader, but to answer the question of whether a machine can imitate a person or not. Although there is some debate as to whether Turing intended this option or not, Sterrett believes that Turing intended this option and thus combines the second option with the third. At the same time, a group of opponents, including Treyger, does not think so. But this still led to what might be called the “standard interpretation.” In this variant, player A is a computer, player B is a person of any gender. The task of the presenter is now not to determine which of them is a man and a woman, but which of them is a computer and which is a human.
Turing in 2012
A special committee has been created to organize events to celebrate the centenary of Turing's birth in 2012, whose task is to convey Turing's message about the intelligent machine, reflected in Hollywood films such as Blade Runner, to the general public, including children. Members of the committee include: Kevin Warwick, Chair, Huma Sha, Coordinator, Ian Bland, Chris Chapman, Marc Allen, Rory Dunlop, Loebner Robbie Prize winners Garne and Fred Roberts. The committee is supported by Women in Technology and Daden Ltd.