Making Simple Robots (2014)
Chapter 4. Robot Friends and Helpers
I am C-3PO, human-cyborg relations. And this is my counterpart, R2-D2.
Much of the research on robots is based on the idea that they can be much more than smart machines—they can also be our friends and helpers. So scientists are always looking for ways to make it easier for people and robots to understand each other. The fire hydrant–shaped character R2-D2 from Star Wars may only be able to speak in beeps and whistles, but fans will tell you the little droid is smart, loyal, and very cute. His companion C-3PO is much more humanoid in speech, appearance, and behavior, but he’s also somewhat less cuddly. That’s why the field of “social robotics” is investigating ways to smooth the interface between organic and mechanic beings.
The effort has several challenges. First, there’s the robots’ artificial intelligence, or AI—the way they think and behave. All robots think like machines. But robots that behave like machines are difficult to talk to. One line of thinking says that figuring out how to give robots a personality will help make them more people-friendly. In 1997, then-MIT grad student Cynthia Breazeal (who went on to found the MIT Media Lab’s Personal Robots Group) created Kismet, the first social robot. Although Kismet didn’t speak a human language, it could make appropriate sounds and gestures that responded to what a person said to it. Kismet’s programming let it learn from conversations and interactions with humans.
There’s also the issue of how robots look. Robots that resemble mechanical tools are efficient but hard to warm up to. So some scientists focus on giving robots a welcoming appearance. One way is to up the cute factor. Robots with child-like features—big eyes, an innocent expression—are easier to relate to. Breazeal at MIT created the robot Leonardo with the Stan Winston Studio, a Hollywood special effects company. Leonardo was a fur-covered animatronic creature that looked like a cross between a puppy and a human toddler. When you talked to Leo, it would search your face with its huge eyes, and express its feelings by pivoting its ears up or back and gesturing with its little arms. At Yale, the Social Robotics Lab uses robots like Pleo, an animatronic toy baby dinosaur, andKeepon, a yellow rubber snowman-shaped dancing robot whose only features are two eyes and a little button nose, to engage children who have trouble with social interactions.
Then there’s the problem of how robots move. A robot designed to interact with humans needs to be compliant—able to adjust its speed and power to avoid causing injury to anyone around it. One industrial robot designed to address these needs is Baxter from ReThink Robotics (Figure 4-1). Baxter’s friendly tablet face swings around to focus on you when you touch its arm. To teach it what to pick up and where to place it, you just move its lightweight arms to the right spot and push a few buttons. And it works at a human cadence, making it safer than high-speed machinery.
Figure 4-1. The Baxter industrial and research robot scales its movements to human speed, so it’s safer to work around. Credit: Rethink Robotics.
In this chapter you’ll design a chatbot—a computer program that may someday be the “brain” of a humanoid robot—and explore the limits of the Uncanny Valley, the spooky place where artificial creatures look too real to be believed.
SOCIAL ROBOTICS LINKBOX
Nao from Aldebaran
Yale Social Robotics Lab
Project: Make a Chatbot Program
Figure 4-2. The finished Scratch Cat sprite
What Is a Chatbot?
A chatbot is a computer program that can talk with people in natural-sounding language. It includes programs that communicate by reading and writing text, and those that can speak and understand verbal commands.
What It Does
Businesses use chatbots to steer callers to the proper extension and information. Websites use them as for customer assistance. They also act as a voice-activated interface for smartphones and other devices.
Where It Came From
Chatbots were originally designed as an intellectual exercise. In 1950, computer pioneer Alan Turing wrote an article for the philosophy journal Mind in which he pondered the question of whether machines think. One way to tell, he suggested, would be an experiment in which a person had to guess whether he was conversing with a computer or a human being. The “Turing Test” has come to be the standard for judging chatbots. In 1990, inventor Hugh Loebner created the Loebner Prize, an annual competition that offers $100,000 and a gold medal to the chatbot that can pass the Turing Test in spoken conversation. A silver medal prize of $25,000 is offered to any program that can fool half of the panel of judges into believing it is human based on a text message conversation. So far no chatbot has claimed the top prizes, but a bronze medal is given to the best competitor every year. In 2012, inspired by a paper that proposed a new Turing Test based on the way children learn to use language, a new Junior prize category was added that is judged by a panel of 12- to 14-year-olds.
The first actual chatbot, Eliza, was a text program developed by MIT professor Joseph Weizenbaum in 1966. Eliza was a take-off on the kind of therapy in which a psychologist turns every response from the patient into a question. For instance, if a user told Eliza they were feeling sad, the computer might respond “Why do you think you are feeling sad?” Eliza was so convincing that test subjects asked to be left alone with “her” so they could talk privately. In 1995, Richard Wallace created an open source program calledArtificial Intelligence Markup Language (AIML) to design his chatbot ALICE, which stands for Artificial Linguistic Internet Computer Entity. AIML is still used today by chatbot researchers and hobbyists.
In 2013, the Loebner Prize bronze medal went to an online text chatbot named Mitsuku whose talents include “Yo Mama” battles with feisty fans. British musician Steve Worswick created Mitsuku as a way to entertain visitors to his site. Mitsuku also won theFunniest Computer Ever competition in 2014, a contest organized by computer science professor Sam Joseph of Hawaii Pacific University with the goal of finding a chatbot able to compete head-to-head with a human comedian.
Perhaps the most lifelike chatbot to date is Siri, the personal assistant built-in to Apple’s iPhone and other products. Siri takes your verbal musings and uses them to control built-in apps like the alarm or music player and to do online searches when you want to find a movie playing near you or book a reservation at your favorite restaurant. But users also like the fact that Siri “knows” a lot about popular culture and can carry on interesting conversations. Siri’s playful personality is so appealing that it inspired the 2013 movie Her, which revolves around a man who falls in love with the voice on his cell phone.
How It Works
To talk to humans, chatbots have to juggle a number of tasks at the same time. First, a chatbot that responds to spoken language must contain some kind of speech recognition software. After all, it can’t give you an answer if it doesn’t hear you correctly. Speech recognition software takes your spoken language and converts it into a digital form. It then analyzes each individual sound to figure out which phoneme (small units of sound that make up the words of a given language) you’re using. English has about 40 phonemes; other languages have more or less. As it identifies the stream of phonemes, it compares them to statistical models to guess how they are most likely to be combined into words and sentences. It’s a difficult task, but speech recognition software has come a long way in just a few years. Still, a personal assistant like Siri can have trouble deciphering accents that vary too much from the standard pronunciation, such as a Scottish brogue, until it “learns” the particular verbal patterns of its user. It does this by keeping track of when you like an answer and when you repeat a command to make sure you’re understood.
No matter whether you’re talking to your chatbot verbally or via text, however, a chatbot must be able to understand everyday conversation. In a computer language, there’s usually only one right way to say something. Any little spelling or missing punctuation mark can throw off the entire program. But humans can express the same idea in countless ways, and new words and phrases pop up all the time. A chatbot needs to figure out exactly what you’re referring to, using clues like context.
Unless a chatbot is just designed to keep you talking, like Eliza, it also needs a databank of information to draw upon. A chatbot like Siri can access both onboard and online information to find the answer you’re looking for. And it can refine the accuracy of its answers through feedback from its own user and from the queries and responses of everyone else who uses the system, too.
Finally, chatbots must also know a little bit about human psychology. Eliza’s open-ended responses were perfectly suited to encourage users to pour out their thoughts and feelings, as they would in a therapy session. But chatbots that can also keep you talking by learning how to push your buttons. In 1989, when the Internet was mainly used to send email and chat messages between colleges, a student at University College in Ireland named Mark Humphrys decided to test out an argumentative version of Eliza he calledMGonz. He set up MGonz to receive his messages when he wasn’t at his desk. An unknown student from Drake University in Iowa who tried to contact Humphrys after he had left for the night ended up trading insults with MGonz for over an hour. According to the transcript Humphrys found the next day, the Iowa student never realized it was a computer he was talking to, not a frustratingly rude computer science student.
WHAT HAPPENS WHEN TWO CHATBOTS MEET?
Figure 4-3. Cornell’s “AI vs AI” video on YouTube
In 2011, Cornell University professor Hod Lipson set up two versions of an online chatbot called Cleverbot on adjacent computer screens and let them talk to each other (Figure 4-3). Cleverbot’s repertoire of responses is completely based on past conversations with humans. Since it typically engages in around 10,000 conversations at once, this gives it an amazing variety of things to say. When Lipson posted the video on YouTube, it quickly went viral. Here’s part of their conversation:
Shruti-bot: Hello there.
Shruti-bot: How are you?
Alan-bot: Great. You?
Shruti-bot: I’m okay.
Alan-bot: That’s good.
Shruti-bot: Are you good?
Alan-bot: Yes, I am good. I just said I was.
Shruti-bot: No, you didn’t. I said I was.
Alan-bot: So you are a robot.
Shruti-bot: No, my name is Cleverbot.
Alan-bot: Yes, you are a robot and your name is Cleverbot.
Shruti-bot: I am a robot.
Alan-bot: Yes, I know.
Shruti-bot: Together, we are robots.
Alan-bot: I’m not a robot. I’m a unicorn.
Shruti-bot: But you said earlier that you are a robot.
Alan-bot: I did not.
Shruti-bot: I thought you did.
Alan-bot: You are mistaken, which is odd, since memory shouldn’t be a problem for you.
ALICE and AIML software
MGonz (Warning—not suitable for children!)
Cornell Cleverbot conversation
Making the Project
You will be creating a text-based chatbot in an open source programming language called Scratch (Figure 4-4). Scratch’s free drag-and-drop software was designed by the Lifelong Kindergarten group at the MIT Media Lab to help children learn the basic concepts of programming. Even though it’s for kids, it’s great for beginning adults, too—and it’s fun! You can create your program right on the MIT-based website, and explore other people’s projects as well. Copying and building upon other people’s projects, or “remixing,” is encouraged. Despite its simplicity, you can create quite sophisticated projects in Scratch.
Scratch 2.0, the current version of the program, can be used online on the Scratch website. At the time of this writing, only Scratch 2.0 Offline Editor and Scratch 1.4 (the previous version) are available for download.
You will not be building a physical robot for this project, although you have the option to change the look of your chatbot’s onscreen avatar.
Figure 4-4. The Scratch home page.
SCRATCH AND OTHER GRAPHICAL PROGRAMS
The beauty of graphical programs like Scratch is that instead of having to type in commands (and worrying about whether they are correct to the letter), all you need to do is move around “blocks” of code on the screen with the cursor. Blocks that can be used in sequence will snap together; try to put the wrong type of command in the wrong place and it just won’t fit. Of course, you still need to figure out which commands to use and get them in the right order, but drag-and-drop programming languages make it simple to switch blocks around and try out new programs over and over.
Drag-and-drop software is seen as a powerful education tool. By lowering the barrier to creating a first program, it attracts students who don’t necessarily consider themselves tech-minded. Carnegie Mellon’s free program Alice—created by Randy Pausch (the late computer science professor whose “Last Lecture” became a viral video and book) is another drag-and-drop program that lets users create animations and tell stories. Its graphics were designed by computer game giant Electronic Arts, Inc. (EA). Alice has been a stepping stone for students going on to learn traditional programming languages like Java.
Graphical software has also been used to program robots, microcontrollers like the Arduino, and other physical computing devices. Lego has long used the drag-and-drop program LabVIEW to control its robotics kits. The downloadable Scratch 1.4 can be used with the Lego Education WeDo kit for elementary school–aged children (with Scratch 2 to be available in the future). It can also be used to control the Xbox Kinect and with MaKey MaKey, a kit that turns any kind of conductive object—like bananas—into a keyboard. Scratch for Arduino, developed by the Citilab Smalltalk Team in Spain in 2010, is a modified version of Scratch that can interact with Arduino boards. A similar program, the open source Ardublock, was designed by David Li of the Shanghai makerspace XinCheJian.
Don’t forget to document your work!
§ Time Needed: 2-3 hours
§ Cost: None (computer and Internet access needed)
§ Difficulty: Easy to moderate
§ Safety Issues: None
What You Need to Know
§ Skills You Already Have: Computer use (logging onto websites, saving work in progress)
§ Skills You Will Learn: Programming concepts, Scratch software
Gather Your Materials
§ Computer and access to the Internet software or online tool
Step 1: List Your Requirements
At a minimum, your chatbot should be able to carry on a short conversation with a human in natural-sounding language. For your first try at programming, that should be enough. Once you’ve got the hang of it, though, you can add features like a personality—funny, thoughtful, obnoxious—or give your chatbot a task, such as playing Twenty Questions or (pretending to) tell the future.
Step 2: Plan Your Project
When you write your Scratch program—or “script,” as it’s called in Scratch—you will actually be creating a little interactive animation, starring a character or object called a “sprite.” Each sprite does whatever you tell it to do—moves, dances, talks (using text balloons), or make sounds (using audio files).The default Scratch sprite is the Scratch Cat, the orange cartoon character that serves as the Scratch mascot and logo. You can also choose a sprite and background from Scratch’s library of available images, or upload your own. Scratch has its own online graphics software that lets you alter any image or draw your own from scratch (no pun intended).
Before you get started, take a few moments to go through the built-in Scratch tutorial (Figure 4-5). Go to the Scratch home page and click “Try It Out” or “Create.” A blank editing page will open up. The step-by-step animated tutorial should be open on the right side of the screen. If you need to refer to it again at any point, just click on the little question mark in the top righthand corner.
Figure 4-5. The Scratch tutorial.
The tutorial will give you a quick idea of what tools are available and where to find them. But there’s still a lot to learn about using Scratch and about writing a chatbot program. As with many kinds of programming languages and programs, the easiest way to learn how they work is to copy someone else’s project and then tweak it to see what happens. In Scratch, the way to do that is to open up any project page and click See Inside.
To get you started, I’ve created a very basic Step by Step Chatbot on the Scratch website that you can use as a model for your own Scratch program. It incorporates a few programming concepts like nesting, loops, and “if” statements (more on those soon). To see how it works, go to the project page for Step by Step Chatbot (http://scratch.mit.edu/projects/19361862) and try it out. Enter different answers to see how it affects what happens. Then click on See Inside and take a look at the program itself. Don’t worry if it doesn’t make sense at this point—you’ll go through the process of putting it together step by step soon. As with all projects on the Scratch website, you can remix my Step by Step Chatbot project and use it as the basis for your own chatbot.
As already mentioned, you can download Scratch to your computer and work offline, but if possible use the online version. Sign up for a (free) Scratch account and you can save your project and share it with the online learning community as well. All it takes is a username, password, and an email address. However, even if you don’t sign up, you can still create a project while you’re online, access the tutorials, and check out the code behind other projects.
SCRATCH CHEAT SHEET
Some quick tips to help you get started.
The Scratch editing page has three columns.
On the left is the Stage where your animation will play. Underneath is the toolbar for the backdrop and for sprites. You can customize them by clicking the icons in the toolbar directly above it. Above the Stage is a blank field where you can enter a name for your project, and above that is the toolbar for the page.
In the center of the page is the Block Palette. It has the command blocks you need to build your project, arranged by category. The category names are not quite intuitive, but they are color-coded, so if you see a command similar to the one you’re looking for you can try the same category.
To to the right is the Scripts Area. To add a command to a script, just click on the block you want to use and drag it over to the Scripts Area.
To add a second command, drag it over to the Scripts Area and nudge it under the first block until it snaps into place. Continue to add commands the same way.
To move a bunch of connected blocks, click on the top block.
To disconnect part of a stack of blocks, click on the topmost block of the ones you want to move and pull them down and away from the rest.
You can change the number or text in the white field of a command by simply typing right over it.
You can layer certain blocks on top of one another. For instance, some blocks have pointy-sided blank spaces on them. Pointy-sided commands or operators (such as “=”) will fit into those spaces. Just drag one block over the other until it clicks in. Once you start building multiple layers, double-check that you are plugging the right block into the right space.
Scratch lets you create a loop, a bit of programming that will repeat over and over until a certain condition (which you specify) is met. The framework of a loop is a block from the tan Control category of the Block Palette with a mouth-like opening. The mouth will stretch around whatever size stack of blocks you place inside it. You can also put one loop inside another—that is called “nesting.” Think of those wooden Russian dolls of diminishing size that nest one inside the other.
You can check your program at any time by clicking on the stack of blocks and watching them play on the Stage.
Step 3: Stop, Review, and Get Feedback
The Scratch website has tons of support, including the above-mentioned tutorials, forums where you can ask questions, and materials for educators and parents. There are also some books that are useful, which I have listed in Graphical Programming Linkbox. One handy teacher-created resource is the Scratch Chat Bot Assignment worksheet.
Step 4: Build Your Prototype
These directions show you how to put together the Step by Step Chatbot. Be sure to check them against the illustrations.
Brackets like this […] indicate a field that gets filled in by you when writing the program, or by the user when running the program.
1. Go to scratch.mit.edu, sign in, and click Create. For now it’s probably easiest to use the default Scratch Cat (the orange Scratch logo). Click File and save your project. This will preserve at least some of your work if you close the page accidentally.
2. Start the Step by Step Chatbot script by going to the purple Looks category of the Block Palette and dragging the block “say [Hello!] for  secs” over to the Scripts Area. Click on the block in the Scripts Area to watch it play.
3. Now for a bit of computer logic! You will give the program some data and tell it what to do with it. Go to the light-blue Sensing category of the Block Palette, and drag and drop the “ask [What’s Your Name?] and wait” block to the Scripts Area. Nudge it under the first block until it snaps into place. Click on the connected blocks to play that bit of code. On the Stage, you will see a field open up where users can enter text. That text—in this case, the name—will then become data which is stored in an “answer” block. The program can then use the data in various ways.
4. To make the Scratch Cat say “Hello, [name]!” you have to layer several blocks on top of each other. Go back to the Looks category of the Block Palette and add another “say [Hello!] for  secs” block to the bottom of the stack of commands.
5. Then go to the green Operators category of the Block Palette and grab an oval-shaped “join [hello][world]” block. The “join” block lets you fit two things into one space. Drag the green oval over to the Scripts Area, and hover it over the purple “Say” block you just added until you see the white field begin to glow. The green oval should then snap into place in the field. Grab a second “join” block and drag it on top of the first “join” block. Snap it into the second white field, the one that says “world.” Now you have three fields to play around with. In the first, capitalize “hello” and add a comma and a space (“Hello, ”). In the last field, replace the word “world” with an exclamation mark (!). In the remaining white field (which says “hello”), you’ll put an “answer” block that will contain the name the user gives you. If you want the user’s entry to be shown on the Stage as the program runs, check the box next to the “answer” block in the Sensing category of the Block Palette.
6. Open the light-blue Sensing category of the Block Palette and drag an oval “answer” block over to the middle field until it snaps in. Tip: Make sure only the field where you want to insert the “answer” block is lit up, not the entire “join” block. You may knock the blocks you already assembled out of place! Click on the stack of blocks and see how your program runs. You should see the Scratch Cat ask you what your name is, and then greet you by that name.
7. Add another “ask” block to the bottom of the stack, and change the text to “Do you want to chat?”
8. Next up is a loop. The loop for the Step by Step Chatbot will be a “repeat until […]” block. Each time the loop plays all the way through, the program will ask whether the user wants to keep chatting. As long as the answer isn’t “No,” the program will go back to the top of the loop and start over. When the answer is “No,” the program will leave the loop and continue on. Go to the tan Control category of the Block Palette and drag a “repeat until […]” block over to the Scripts Area, but don’t connect it to the stack of commands yet. You’ll build and test this bit of code separately, to make it easier to work with. The condition needed to end the loop is for “answer” to equal “No.”
9. In the green Operators category of the Block Palette, you can find a pointy-ended block with two blank fields and an equals sign (“=”) in between . Drag it over to the empty field on the “repeat until” block and snap it in. Then go to the light-blue Sensing category of the Block Palette and drag an “answer” block over to the first empty field of the “equals” block. Type “No” in the second empty field.
10. Now get a little conversation going back and forth between the chatbot and the person using it.
o From the light-blue Sensing category of the Block Palette, drag an “ask and wait” block into the mouth of the loop. Change the question to, “OK, what would you like to chat about?”
o Drag a second “ask and wait” block underneath the first. Change this question to “Do you like [answer] a lot, a little, or not at all?” To do that, you’ll need to layer green “join” blocks and drag over an “answer” block the same way you did in #3 above. Don’t forget to include spaces in the text that comes before and after the “answer” block. (By the way, when computer programmers want to avoid writing out the same set of instructions in more than one place, they tell the computer the equivalent of “do what you did in #3.” That’s called a subroutine.) Run this stack to see if it’s working before going on.
11. To analyze the user’s answer so the program knows what to do next, you’ll use another kind of Control block: the “if-then-else” block. You may remember if/then statements from high school geometry. They work exactly the same way here. If a certain condition is met, then the program takes one course of action—or else it takes another action. Since you asked the user to choose from among three responses, you’ll need to give the program directions for all three. In this case, the answers “a lot” and “a little” both lead to the same course of action. Here’s how to assemble that bit of code:
o Drag an “if-then-else” block to the Scripts Area from the tan Control category of the Block Palette. Don’t connect it to any other stacks of blocks yet.
o Drag a green “or” block from the Operators category of the Block Palette and plug it into the pointy-ended hole between “if” and “then.”
o Drag green “=” blocks into the holes in the “or” block. Make the first one “answer” = “a lot” and the second one “answer” = “a little.”
o Drag light-blue “ask and wait” blocks into each mouth of the “if-then-else” block. Type “What do you like most?” into the first and “I’m sorry to hear that. Why not?” into the second.
o Drag the “if-then-else” block into the mouth of the “repeat until” block and plug it in under the two “ask and wait” blocks. You can now test the loop by clicking on it to see how it runs.
12. Feel like drawing out the conversation? Add another leading question that will keep the person using your chatbot program talking. For instance, you can drag an “ask and wait” block under the “if-then-else” block, making sure it is still inside the mouth of the “repeat until” loop. Then type a line of dialogue such as “Really! Why do you think that is?”
13. Time to give the loop something to judge. The last piece of dialogue in your loop should be a yes or no question that will signal whether or not the person using it is ready to stop. Drag an “ask and wait” block below the last one (still inside the “repeat until” loop). Insert a comment such as “I totally agree! Is there anything else you’d like to talk about?” If the answer is “No,” the program will exit the loop. Any other answer will signal the loop to repeat again.
14. End the conversation with a little good-bye message. Drag a purple “say [Hello!] for  secs” block to the Scripts Area and attach underneath the loop. In the text field, type in “OK, thanks for dropping by!”
15. Drag the entire stack of code over to the first segment you wrote and attach them. As a final touch, add a “when clicked” block from the brown Events category of the Block Palette to the very top. This tells the program to run when a user clicks the green flag on the project page. You can add a tan “stop [this script]” Control block at the end, but it’s not really needed in this program. Your chatbot is now finished and ready for its first tryout!
BREAK THE CODE: “HELLO WORLD”
When you’ve just put together a new electronics device or started working in a new computer language, the “Hello World” program is the first thing you try to make sure it’s working right. The phrase is generic: it doesn’t have to literally make the machine say “Hello World.”
Step 5: Test Your Design
Test your chatbot a few times yourself by running through the project, entering different kinds of answers. If you need to fix a line or want to add a step, this is the time to do it. Then invite friends to try it out. If you create your project on scratch.mit.edu and set it to Share, they can access it from their own computers.
Step 6: Troubleshoot and Refine
It can take a little while to get the hang of dragging and dropping the blocks just where you want them. See Scratch Cheat Sheet for help.
As for refining your program, that’s up to you. Once you’ve got a basic chatbot working, you can add as many levels of complication as you like. For instance, to keep things simple, the Step by Step Chatbot requires the user to type “No” to end the program—but that’s not spelled out in the dialogue boxes. There are a number of ways to solve that dilemma—including accepting more types of negative answers, or getting more clever with the conversation. Take a look at the code for more advanced chatbots like theGLaDOS chatbot on the Scratch website (see link below) for inspiration.
Step 7: Adaptations and Extensions
This project can be adapted for use without a computer by simply writing it out on a piece of paper as a logic tree—kind of like those old Choose Your Own Adventure stories. Start at the top of the page with the first line of dialogue, and then branch off below each time the user gets to make a choice one way or the other.
You can also extend the project by giving your chatbot a physical body. Load it onto a phone or tablet (or a Raspberry Pi) and build a robot body to hold it.
GRAPHICAL PROGRAMMING LINKBOX
MIT Scratch website
Scratch 2.0 beta download
Scratch 1.4 download
Carnegie Mellon’s Alice
Scratch: How to Connect to the Physical World
Scratch for Arduino
Step by Step Chatbot
Scratch GLaDOS chatbot
And in print…
§ Learn How to Program with Scratch by Majed Marji (No Starch Press, 2014)
Project: Picture Yourself in the Uncanny Valley
What Is the Uncanny Valley?
Figure 4-6. An Uncanny Valley face.
The Uncanny Valley is a theory that says people feel more comfortable with robots that look more human—but at a certain point, an artificial being that’s just a little bit off becomes frightening or repulsive.
What It Does
Fear of falling into the Uncanny Valley makes it hard for designers in the fields of robotics and computer animation to make realistic-looking representations of humans.
Where It Came From
The Uncanny Valley was first described by Japanese roboticist Masahiro Mori in 1970. He suggested that a line graphing the steady increase in how much people liked humanoid robots as they became more realistic would take a sharp dip right before reaching the point where they were indistinguishable from actual human beings. At a robotics conference in 2013, Mori said he was inspired by his work on prosthetic limbs. He noticed that people would respond differently to an artificial hand that looked like a tool or machine as opposed to one that attempted to look like a real hand.
Figure 4-7. A simplified version of Mori’s Uncanny Valley diagram. Credit: Karl MacDorman under a GNU Free Documentation License.
Of course, at the time Mori first developed the Uncanny Valley concept, robots weren’t very realistic looking. So it took another 30 years for the idea to seep into popular culture. One inkling came in 2001, when children in test audiences for the 3D animated filmShrek began to cry at the appearance of a quasi-realistic Princess Fiona. That same year, a film called Final Fantasy: The Spirits Within, based on a video game, bombed at the box office in part because its computer-generated characters gave off a weird vibe. The 2004 movie Polar Express, which featured a motion-capture animated version of actor Tom Hanks, gave the term even more exposure. Today the idea of the Uncanny Valley is a factor that designers keep in mind when working on video games, computer graphics, 3D animation, and plastic surgery.
Robots balancing on the lip of the Uncanny Valley affect are most common in Japan. In 2011, Showa, Waseda, and Kogakuin Universities worked with the robot maker Tmsuk to create a simulated dental patient for the Yoshida Dental Manufacturing company.Hanako 2 had soft silicone skin, a tongue that moved, eyes that blinked, and a head that turned in response to a painful touch by the dental student. And roboticist Hiroshi Ishiguro of Osaka University in Japan made a robotic clone of himself named Geminoid. A former painter, Ishiguro started out building robots that looked more like trash cans or insects, but got a mixed response. So he decided to see how people would react to a nearly human robot (Figure 4-8). His silicone-skinned robotic doubles often wear the same serious expression as their maker. To add to the realism, he even tops them with hair from his own scalp.
Figure 4-8. Hiroshi Ishiguro developed Geminoid HI-4 in his own image to explore whether human presence could be transferred to a remote location using robots. Credit: Hiroshi Ishiguro, Osaka University.
Meanwhile, in the United States, David Hanson of Hanson Robotics creates robots that both look and act like real human beings. Hanson’s android version of the science fiction author Philip K. Dick (whose book Do Androids Dream of Electric Sheep became the movie Blade Runner, featuring robots that don’t realize they aren’t real people) can autonomously answer an interviewer’s questions while making appropriate facial gestures. Hanson has said he’s constantly refining his Philip K. Dick android, and he’d love to see it star in its own television talk show.
How It Works
No one is sure how the Uncanny Valley works. In fact, there is disagreement about whether it really exists at all. In 2005, David Hanson wrote a paper disputing the Uncanny Valley, claiming that test subjects had no problem with his lifelike robots. A year later, Karl MacDorman, professor of Human-Computer Interaction at the Indiana University Department of Informatics, compared the reaction of observers to a video clip of a stylized robot morphing into Hanson’s Philip K. Dick robot and then into a photo of the actual author. His results suggested that other components, such as how strange or familiar a robot looked, also played a role in determining how much of an Uncanny Valley effect was felt.
But other studies seem to show a biological component to the reaction. A study done by neuroscientist Asif Ghazanfar at Princeton University in 2009 found that monkeys would avoid looking at photos that were close to but not exactly monkeys, compared with photographs or symbolic drawings. That may mean that the Uncanny Valley came about through evolution—perhaps because it alerted our ancestors that something was “off” about an individual whose appearance was different.
Motion may be another contributing factor. The body of the headless Big Dog robot from Boston Dynamics looks more like a keg of beer than a Doberman. But its legs run in a very dog-like fashion, which can be unnerving to watch. Likewise, passive-dynamic walkers—robotic sets of human-like legs that many labs study to determine how gravity can help make robots walk—are spooky to watch as they casually stroll, bodiless, down an incline. And on a commonly used chart of the Uncanny Valley, zombies (“the living dead”) rank considerably lower than an ordinary corpse.
Hiroshi Ishiguro’s goal is to create a robot that could pass the visual equivalent of a Turing Test and fool a human into thinking it is real. But Masahiro Mori himself has said he believes a better solution to the Uncanny Valley problem is to work on friendly designs that are not quite so real.
Making the Project
For this project you will explore the limits of the Uncanny Valley by making as close a likeness of yourself as you can. Will the result be cool or creepy? Survey your friends to find out.
Don’t forget to document your work!
§ Time Needed: 1–2 hours
§ Cost: $10 or less
§ Difficulty: Easy
§ Safety Issues: None
What You Need to Know
§ Skills You Already Have: Playing with modeling compound
§ Skills You Will Learn: Making a lifelike model
Gather Your Materials
§ Two copies of a black and white photo of your head, facing forward, against a white background, as large as possible (standard copy paper is fine)
§ Waxed paper or tracing paper, about the size of copy paper
§ Masking tape or glue stick or spray glue
§ Poster board or other smooth stiff board
§ Crayola Model Magic or other modeling compound, in skin tones or white
§ Washable markers for coloring the modeling compound
§ Clay modeling tools or kitchen utensils, including:
§ Unsharpened pencil with clean eraser
§ Craft sticks
§ Plastic forks, knives, and spoons
Step 1: List Your Requirements
The goal of this project is to explore the Uncanny Valley by making a lifelike copy of your own face. To add to the realism, you will make a bas-relief model using modeling compound to raise the design a little from the background and create lights and shadows. You should use a stiff lightweight base if you intend to display it at an upright angle.
Step 2: Plan Your Project
The key to creating a model that looks as lifelike as possible is to start with a good photo. The more prominent the outlines of each feature, the easier it will be to extend them into the third dimension. Think about what other details will help your clone look eerily believable, and try to include as many as you can. For instance, individual teeth inside a mouth can look “too real,” so try taking your photo with your mouth open or a big wide smile. But any wrinkle, scar or other imperfection will help move your model from artistic to hyper-realistic.
Like roboticist Hiroshi Ishiguro, you can also make your Uncanny Valley face seem more real by giving your double your own personality and emotions. Or instead of a friendly smile, use an exaggerated expression, like a robot that doesn’t quite get social norms for human interactions. If the thought of building your own clone is too unsettling, try using someone you know in real life as a model, so you can tell what their features look like in three dimensions. Failing that, try to use photos from at least two views, from the front and from the side.
Step 3: Stop, Review, and Get Feedback
For ideas on how to take a robot face from cute to creepy, check out photorealistic artists whose paintings and sculptures give viewers those same uneasy feelings as humanoid robots. For instance, in 2013 sculptor Tony Matelli’s life-size, full-color statue of a man sleepwalking in his underwear upset students when it was placed on the grounds of Wellesley College in Massachusetts. The man’s zombie-like pose, bald head, pale skin, pot belly, and droopy drawers (as well as the fact that the nearly naked figure was situated on the grounds of an all-women’s college) heightened viewers’ feelings of disgust and fear.
Tips on using Model Magic or similar modeling compounds: Crayola Model Magic feels “foamy” and is easy to mold. It air dries quickly and becomes permanent overnight, but retains its rubbery appearance. Buy the smallest pouch or container you can, as once it is opened it should be used within a few days. Store opened packages in an airtight container, like a ziptop bag. As you work, use the smallest amount possible. It’s easier to add more than to take away extra. One secret to making your model look real is to build it up in layers, letting them overlap, and then using your carving tools to create a little overhang. So the teeth should overhang the inside of the mouth, and the lips should overhang the teeth. That will create shadows that give your sculpture the illusion of depth.
Step 4: Build Your Prototype
1. If you want to make a model you can display, attach the photo you are working from directly to the board you are using for a base with a glue stick or spray glue. Otherwise, you can just tack it down with small pieces of masking tape at the corners. You may want to attach a sheet of waxed paper or tracing paper over your photo with masking tape that you can lift to check how closely you are following the guide photo underneath the modeling compound.
2. Model Magic offers a set of natural colors that work well for skin tones, but you can mix your own custom shades with washable markers. Try to mix up as much as you will need all at once, to keep the color consistent. To tint white Model Magic, take a hunk and flatten it somewhat into a thick pancake. Then take the tip of a washable marker and roll it back and forth on the pancake’s top. When you’ve got about as much pigment as you think you’ll need, start smushing and kneading the compound until the colors are evenly blended.
3. Start building the face by working on the highest parts first—the nose, chin, cheeks, and brow. Use a plastic spoon and other tools for smoothing the skin and digging creases where they appear on the photo. Remember that creating “overhangs” in places like the nostrils adds realistic shadows. And pay attention to the shape of the nostrils, because they are unique for each person and add to the realism. Refer to the second copy of the photo as a guide.
4. Next work on lower spots like the throat, mouth, and forehead. Build up the face piece by piece, connecting them as you go. For the ears, start with the highest parts, then fill in the flats. Again, the curves of the ear are unique, and the opening to the ear canal is another place to create an overhang that makes the model look realistic. Fill in any other skin areas, leaving the mouth and eyes open.
5. If the mouth in the photo is open enough to see the tongue or gums, mold that first. Make each tooth individually with white Model Magic, using your tools to shape them and keep them separate. For the eyes, start by making a flattened ball for the iris. Add little bits of white a little lower, tapering down at the corners to indicate the curvature of the eyeball. You can use a black marker to draw on the pupils. Then add the eyelids and eyelashes.
6. With your lip color, add lips overlapping the teeth. Be careful not to mix the white with the surrounding skin color.
7. Making believable hair from Model Magic is difficult. For eyebrows, eyelashes, as well as hair on the scalp, indicate individual strands by using a fork as a carving tool. Add hair to the scalp in clumps, to give it dimension. Alternately, you can let your model dry overnight and then glue on artificial or real hair.
Step 5: Test Your Design
To see how successful you were with your model, take a survey of your friends. Ask them to rate it for factors such as how recognizable it is and how realistic. To measure the Uncanny Valley effect, also ask how it makes them feel—are they happy to see it, or does it make them feel weird?
Step 6: Troubleshoot and Refine
Unfortunately, if you’re working with Model Magic, you basically will only get one shot to get it right. The compound is easy to work with, but it dries quickly. You’ll need to complete your model in one session.
Be careful while you’re working not to smush up a part you’ve already finished. Be especially aware of areas where different colors meet and try to keep them from running into each other.
Because Model Magic has a rubbery texture, it will add realism to your model’s skin. But some areas, like the eyes and teeth, will look more believable if they appear slightly moist and shiny. You can highlight those areas by brushing on some Model Magic Glossy Glaze or other shiny finish coat when the modeling compound has dried. You can also dress up your model with artificial hair, or give it accessories such as glasses or earrings.
Step 7: Adaptations and Extensions
If creating a 3D version of your face from a flat surface is too hard, you can use a three-dimensional mask (art retailers have blank ones) or head form like those used to model wigs and build it up with the Model Magic.
You can also make the project simpler by keeping it two-dimensional and just tracing your photo onto tracing paper. Even a 2D line drawing can approach the Uncanny Valley if you include enough details. Or just focus on one feature, such as an eye or a mouth, that can in itself evoke an Uncanny Valley reaction.
As an extension, use your Model Magic doppelganger (or your actual face) as a mold to make a silicone mask. The Smooth-On company has kits and friendly tech support people to help you get started.
Or go digital and use panoramic photos and CAD software to make a 3D computer model of your face. You can then turn it into a pattern to build with cardboard or print it out on a 3D printer.
UNCANNY VALLEY LINKBOX
Masahiro Mori’s Uncanny Valley paper “Uncanny Valley Revisited” robotics summit
Hanako 2 dental robot
Hiroshi Ishiguro Laboratories
Crayola Model Magic
Smooth-On silicone lifecasting materials