Meet New Pepper--Robot with Emotions

By Keisuke Tanabe : Reporter of Toyokeizai
July 06,2015
Keisuke Tanabe
Reporter of Toyokeizai

Writer and Editor of Toyo Keizai Inc. since April 2007. He previously covered retail industry, writing feature stories such as "End of Traditional Retailers".  He also worked as an editor of Toyo Keizai's investment informantion magazine. His present interest is in the telecom industry and mobile technology.

Pepper the emotional robot went on sale on June 20(Photo by Shuji Umetani)

SoftBank CEO Masayoshi Son was nothing if not talkative during the briefing held on June 18 in advance of the public release of Pepper, SoftBank's humanoid robot.

Son emphasized the advances his company had made with the product, stressing that "by equipping Pepper with an emotion engine, we have made him capable of having his own emotions." Son's enthusiastic explanations and responses put him far over the allotted questions-and-answer time.

Kaname Hayashi led the development team

Since the original unveiling of Pepper in June of last year, development efforts for this project have picked up steam at SoftBank Robotics as well as participating third-party developers.

Leader of the development team Kaname Hayashi has pinned his hopes on the accelerating pace of development activities surrounding the robot.

He has showcased Pepper at various event venues and involved the robot in unexpected collaborations, telling us that "some developers continue to pursue development efforts regardless of the costs, and we are seeing the unprecedented levels of enthusiasm and excitement we had hoped for. For this I am truly thankful."

Differing approaches for home-use versions

The development team's one-year journey culminating in the start of home-use Pepper sales was anything but easy. Following the announcement of the product in June of last year, Peppers were put to work greeting and interacting with customers at major SoftBank shops, and although customer responses were highly positive, the development team found themselves unsure of how to move forward.

Their problem stemmed from the vast differences between a Pepper who could communicate with and entertain in-store visitors for roughly 10 minutes at a time, and a Pepper designed to play a vital, everyday role in people's homes.

The store-use Pepper interacts with customers by telling jokes, chatting, performing robotic dances and the like, whereas the home-use pepper needs to be capable of understanding whether or not the person in front of him feels like having a conversation rather than simply talking to them by default.

Hence the question on the minds of team members: in what ways should Pepper communicate with people, and how could they design him to be a natural presence in the home environment?

This was a major challenge for the team because they would have to program and fine-tune their robot without previous examples to fall back on.

No mechanism existed to enable Pepper to intuitively notice people based on sounds and other such indicators while simultaneously picking up on the general mood in the room like a human would, so such mechanisms would have to be created from scratch. The complete retooling of the product from the ground up turned into a series of trial-and-error experimentations.

Team members racked their brains over the question of how to make a robot seem more like a member of the family. For example, the household version of Pepper includes an app that recommends different recipes when a user wants to prepare a meal.

However, because Pepper is not designed to explain the preparation process in a detailed, step-by-step manner like a recipe website, the team designed him to instead propose recipes in a lighthearted and enjoyable manner so that users would be inclined to turn to him again when they needed a meal recommendation.

Pursuit of a lifelike robot

Even though a large number of intricate functions were originally planned for Pepper, the team eventually scratched this approach.

"We don't demand advanced functionality from our family members or our pets," points out Hayashi. "Caring for a pet makes you feel needed, and teaching a dog to do tricks such as shaking your hand is enjoyable and makes you feel more valued as a person. Rather than designing Pepper as a device that performs functions in place of something else, we decided to make him capable of doing unique and interesting things."

Fumihide Tomizawa, President of Softbank Robotics (Photo by Koichi Imai)

The most important factor when designing a household robot such as this is to ensure that users will not tire or grow bored with its presence.

"This was our main focus with Pepper," explains Fumihide Tomizawa, President of SoftBank Robotics. "In addition to increasing the number of available apps, we programmed various conversational techniques and gestures, made sure his reactions were highly varied, and took every measure possible to keep things interesting."

This approach is closely tied in with the pursuit of a robot possessing its own emotions—the goal so fervently promoted by Son. Home-use Pepper is equipped with a greater capacity for self-motivated actions, performed regardless of the presence or absence of instructions from the user. This functionality is the result of efforts to create a more lifelike robot.

The original Pepper often made on-the-spot decisions based on the current circumstances and situation, but the new Pepper recognizes each family member individually, makes records of their actions and behavior, and stores these records for reference.

In this way, he is designed to develop his own personality and behaviors. "The home-use version of Pepper will be bright and cheerful if treated with affection and complimented often," says Son, "whereas he will become lonely and depressed if he is ignored or neglected by members of the household."

Pepper was supposed to be on sale from February, but the release date was pushed back to June (Photo by Shuji Umetani)

Pepper's original release to the public was planned for February but later pushed back to June due to changes to the CPU that delayed production start-up longer than expected.

The new CPU subsequently taken under consideration had a processing speed four times that of the standard type, but its introduction would mean that preparations for mass production were unlikely to be completed in time to meet the originally scheduled release date.

SoftBank decided on a limited release in February of 300 second-generation Pepper units (containing the new CPU) for developers only, and postponed sales to the general public. "The fourfold increase in performance instantly expanded the range of possible applications and functions for our robot,"says Hayashi.

Future plans

Units began flying off the shelves following the long-awaited June 20 public debut. With encouraging reports coming in, including one of 1,000 robots selling out in a mere minute during the first month, future sales looked promising.

SoftBank started to produce and sell these robots at pace of more than 1,000 units per month starting in July. Furthermore, steady efforts are underway toward the establishment of a global sales framework for Pepper: Among the various measures being taken, Alibaba Group, the world's largest e-commerce corporate group, and Hon Hai Precision Industry, the world's largest electronics manufacturing services provider (EMS), have each made 20% investments in SoftBank Robotics.

Collaboration with IBM Corporation's "Watson" artificial intelligence system is also under consideration.

But development of the Pepper line of robots if far from over, and work on the third-generation's robotic body has already commenced.

"At this point, we have tried out a range of new things, including the emotion engine,” says Hayashi who is determined to make further improvements. “We've conveyed the general, overall direction of where we want to take things, but we still haven't put it all together in a clean and organized way, so many areas still need work. However, I believe you can expect great things from my team in the future, including updates to the current model."