Unite 10: Jesse Schell And The Future Of Virtual Characters

At game engine company Unity's Unite 10 conference in Montreal, Schell Games founder Jesse Schell delivered a keynote called The Future of Virtual Characters, in which he predicted the potential future of interactive characters and the technology requirements to achieve truly compelling and enduring characters. His company is currently developing multiple games, including The Mummy Online, a free-to-play MMO which uses Unity. "I think there's a lot of reason to believe that a lot of the innovation in the next five to 10 to 20 years will be happening on the Unity platform," Schell said, which is what attracted him to speak at the conference. While he admitted he's speaking about the future, "making concrete predictions about the future is the best way to make predictions," he said -- in other words, the more he does it, the better he gets at it. Rolling back to the past, even before games, he said "there's something deep within us that likes the idea of virtual characters." And though we "we often think of virtual characters as something for children," they don't have to be. Schell used to work for Disney, and has had plenty of chances to observe kids interacting with the characters in the suits at the theme parks. Many kids love them, but for the ones who do not, they often react deeply negatively. "There's always this moment where kids are uncomfortable around these characters... and the kids become explosively angry at the characters. Some kids become very upset, and what they become upset about, as far as I've been able to figure, is that they figure out that this character isn't real, and the fantasy has been taken away from them." So how do we preserve this fantasy? Facial expression tracking is one of the core tools -- "the reason this stuff matters is because everything's getting a camera on it." Once devices can track the user's expression, you can map that onto a character and improve interactivity. "You can imagine that in MMOs five to 10 years from now, that every character will map the facial expression, the eye movement of the player who's playing." Schell also sees this as potentially useful for avatar-based teleconferencing, which could well replace video services like Skype, in his view, because it will be more freeing and also improve interactivity. "Once the technology is there to make decent facial expression... It will be an interesting experience to see who uses which avatar when," he said, referring to professional versus social and game situations. At this point, Schell found it important to point out that while there are plenty of skeptics about these ideas becoming prevalent, disruptive technologies often leapfrog those that have slowly and steadily developed over time and gained huge audiences. Using a chart from the book The Innovator's Dilemma, by Clayton M. Christensen -- "everyone should be familiar with it," he said -- he illustrated this concept. "The reason they blow it because they make the terrible mistake of listening to their customers." The customers end up switching to disruptive technologies, "and I think facial avatars will be one of those." Now, you have an interactive avatar -- what would improve that? How about a memory? Persistent databases can allow for this. It's-A You Referring from 1981's Donkey Kong to today, Schell made this point amusingly: "You think of game characters kind of like your friends, but I'm sorry Mario, you're a terrible friend. Mario, dude, we've been together 30 years. I remember you! You don't remember me!" "We have persistent characters on per-game basis, but why not do it on a per-character basis? When you sign into a game he could remember you, show you videos of what you've done before. We could switch from 'It's-a me, Mario!' to 'It's-a you, Jesse!'" Of course, he pointed out, "If we're going to have a long term relationship over time," there's an important stipulation. "We change. Will the characters need to change too?" Speech recognition is going to be crucial to the evolution of characters, too."It's another one of those below-the-line [of necessary-for-adoption quality] technologies we've given up on," said Schell, but "it's crucially important for us. It may be the most important technology for us of all as game designers." USC professor Chris Swain, said Schell, draws a parallel between games and movies here. "People did not take movies seriously when they were silent; they were not serious art or serious entertainment. Then they learned to talk and everything changed. And film and video became the literature of the 20th century. "He suggested we'll see the same parallel with video games. Of course, they can talk, but they can't listen. Once they listen to us and we can have a conversation, games will become the most important medium of our time," said Schell. And for that, natural language understanding will be key. However, he found it important to point out that discussions of this topic are often framed in terms of the Turing Test -- but this "trips us up a lot because there are a lot of things that wouldn't pass that test that still would be interesting to talk to." Text adventures were once popular, but Schell thinks it's not just the rise of graphics that killed them. "The text gives you too many options. It opens it up and says 'type anything', but there are only a small number of actions it can respond to. It's acting like it's free, but it's not, and that's what frustrating." Scribblenauts is another example of this technology -- still incipient, but promising, said Schell. Facial sensing was already covered. Emotions are easily recognized by humans, but computers must be part of that, said Schell. "Once we can do that we can sense your emotions," said Schell, developers can create "a game where you actually have to act, or feel emotions. A game where someone tells you where there dog just died and if you can't manage to cry then no, you're not getting to the next level!" Joking aside, that would leave room for "characters who can sense your emotion [and] may want to act differently." Of course, the best-known example of this so far is Lionhead's Milo demo. "A lot of people were frustrated with this because parts of it were cool technology, and other parts of it seemed to be created out of vapor and lies," Schell admitted. However, "Setting that aside, this is the dream of face-to-face communication with a character... I'm sure that soon it will be very possible." Here's a question: Who will we talk to? Schell's answer wasn't what you might expect. "I think this will fundamentally change the way video games work," he said. While the natural assumption is that it will be NPCs and other players -- similar to contemporary game interactions -- "I think we're going to be talking to our avatars. Normally we think that's crazy -- I control the avatar. I would never talk to the avatar." We already disengage from our avatar in cutscenes, however, he pointed out. Mario talks, and then the player resumes control. This is a process -- "a natural projecting yourself in, projecting yourself out. As such we will develop a relationship with these guys to the point where we may think of them as virtual companions. We may have a relationship with them... and if they have some intelligence they may do some stuff when I'm not logged in." And as games map to more and more platforms, the character (with its persistent database) will hop from one to the next and continue your interaction. "The number of technologies that run 3D graphics is going to continue to diverge. It's not going to converge because technologies never do that," said Schell. These characters could connect to everything from Facebook to GPS, access your health data, your social graph, examine your emails and online purchase info, of course interface with game data, He even envisioned a conversation with a character while driving -- the character continues to play in the persistent game and gives you updates while you're unable to interact beyond speech. Anti-Social Weirdos Schell also sees these characters as potential "cognitive tutors". Research suggests that "if you create a situation where a student's job where it is to teach the virtual character," they perform better. An application called Betty's Brain invited kids to teach a simple character. "They found that when children are trying to help another character they learn better and focus more." The people who are told that they are teaching a character, it seems, "spend more time getting ready, prepping, and end up learning better, because it's something people take seriously." There are currently programs which invite kids to read to dogs at libraries -- because when kids are reading by themselves they read differently than when they read aloud; meanwhile dogs are not judgmental, allowing the kids to learn and improve at their own pace. An important tech that needs to be developed for this all to come together is "intelligent actors". Schell pointed forward to the conversation game Facade as the best example of this. While its AI characters have traditional goals and behaviors as in many games, they're also aware of the story arc of the interaction -- and change their behavior and tone as it's reaching its climax. The developers call this "the value arc". Schell concedes that this is a very complicated area of development, even as he deems it necessary. Augmented reality is another tech he thinks will be important. "One could assume all of these interactions could happen indoors, but there's a lot of reason to believe that soon we can have outdoor virtual experiences," he said. Moving off the screen, even to glasses which overlay virtual characters onto the world, is an area he expects to grow. In the end, these characters will "be an important part of our social life because they're going to be the one you can always go to." Persistent, growing, and fully interactive, "these characters will serve as meaningful emotional crutches in our lives." Schell asked, "is this going to be a good thing? Is this going to make us into a bunch of antisocial weirdos?" He's not sure, but seems to think the positive will outweigh the negative. And while researchers are working on some of the pieces of this puzzle, "the game designers are going to make this first," said Schell. "Think about how Facebook became a game platform -- because 'I'm already at Facebook' ... because people got the Facebook habit, games became a Facebook thing," he said. By the same token, once game characters begin to reach some of these technological areas needed for meaningfully interactive as Schell describes it, "if you're already interacting with [the virtual character] for games... it will become a habit." "What starts as a game could become part of people's lives... And while it may have negative aspects, having friends who are always there for you might not be such a bad thing," he concluded.