Q&A: Google's chief game designer explores the potential of virtual reality

Noah Falstein’s career runs the gamut of game development. In the early 80s he worked at Williams Electronics as co-designer on the arcade game Sinistar, programmed and developed games for the Commodore 64, co-designed highly-influential LucasArts’ Indiana Jones adventure games, played as an advocate for games for health and serious games, and overall has been an active member of the game dev community in his decades of professional life.

Now, Falstein is chief game designer at Google, where he worked on Google Glass and is currently focusing on VR with the Google Daydream team. While he couldn’t talk about specific Daydream projects at Google, he had plenty to say about his influences, game design, and VR.

(Edited for length and clarity.)

I was looking back at some of the old Game Developer magazine articles that you had written and there was an interesting one where you talked to fellow game developers and designers about influences and how they started in game design. I'm just wondering, these days what kind of influences do you have, both game and non-game, that you're thinking about and pondering as a game designer?

I'll quote Jesse Schell who's a fellow game designer I really respect. In some of his writing he's talked about how if you want to really be creative in a field, you have to constantly be referring to things totally unconnected to that field. He gave an example of learning to juggle and someone who watched a flight of birds or things drifting in the wind as an inspiration for his juggling rather than looking to other jugglers.

I think that that really applies to games and game design. It's very helpful to look at a range of things that come from other areas. Recently, I've been doing a lot of work in virtual reality, working with our Daydream Team, and one of the things I've tried to do is to not just focus on other VR projects, which are of course all in early phases comparatively speaking, but to look to other sources like theatre and improv, for example, as inspirations for what might be interesting in VR. Or how movies do closeups and play around with the grammar of filmmaking, and what life lessons we might be able to learn from that, that can apply or might be broken by VR.

Speaking of theatre can you talk about what has influenced you? What have you learned from that?

One of the things I think is really fascinating about VR is that it does a better job of making you the viewer feel like you're in the midst of the action than any type of video-oriented media that we've had before. In that way, it's a little bit more like some of the theatrical productions that mix the actors and the audience together, so you're never really sure if the person next to you is part of the show or just another viewer like you. VR, with its sense of presence and the immediacy of putting you right in the middle of the action, and being able to turn your head to see [the action] instead of looking at a screen, it triggers, I think, a lot of the same feelings that way.

I've been reading up on some of the neuroscience behind it and apparently there are a bunch of neurons that fire only if there are people within arm's reach in our personal space. I think that VR is likely to be shown to trigger a lot of those same neurons in ways that don't trigger when you're only looking at things on a screen. So specifically with theatre, one of the things that I've found is having drama be enacted within that very close arm's-reach distance seems to evoke a much stronger emotional response than anything I've seen in most standard media.

Specifically, one of the things that clued me into this was not a game, but a VR story that our Spotlight Stories group did called Pearl. It was viewed at the Tribeca Film Festival earlier this year. It is a story that's told completely from the vantage point of the passenger seat in the front seat of a car and you're looking at the driver and people in the back seat that are all really close to you. Occasionally the action moves outside of the car and you're looking out windows. But I've been surprised what a strong emotional impact that had for me. I think this research I mentioned is giving some clues as to why that's happening. So it's made me think -- this is all quite recent, just within the last few weeks -- what can be done in VR to take advantage of that really close proximity? That's kind of the direction I'm in. I don't have a lot of specifics for you yet because it's really just so exciting and fresh and new.

People have talked about how VR can elicit empathy. How much have you looked into that and explored how we can appreciate other people's positions?

Well, I haven't done a lot of work on that myself, it's more been observational. The data points that I find most interesting are -- one was the Pearl Spotlight Story I mentioned. Another one that's been widely reported was the work of Chris Milk who's done some VR documentaries. In particular he did some VR documentaries about refugee camps and then was able to show them in VR to people of the United Nations. This is all quite anecdotal, so the scientist in me is a little skeptical until we have more data. But at least in terms of people's verbal reactions, they found these experiences in VR to be in many ways much more gripping than either hearing stories, even firsthand stories, or watching traditional movies.

Having watched some of those myself, one of the interesting serendipities that they got out of it is that when they took a 360-degree camera into refugee camps, not surprisingly all these refugees would turn and stare at this bizarre technology that nobody had seen before, with all these cameras pointing every different direction.

As a consequence, when you watch the 3D film that they made out of it, you see all these people who are turning and looking at you, often with curiosity, sometimes with fear or even confusion. I think that they benefitted from that because it makes you feel as if you're really there having people respond to you, even though they're really responding to the camera. So that was an interesting accident. I expect, as with a lot of other media, many of our early breakthroughs will be through chance or accident rather than through design and we just have to be very sharp about picking up on those when we can and realizing what the implications are.

You wrote a really interesting article on Gamasutra about Google Glass, and it was about things to consider when developing with Google Glass. You had named three core concepts that Google tries to stick to with Google Glass: humanity, immediacy, and simplicity. Does Google still hold those ideas for AR and what are some of the core concepts that are applied to VR at Google?

Certainly those three concepts were specifically described for that one project, and in general I think they're good, solid concepts that apply everywhere. I haven't heard that same set of principles applied to other concepts like our VR. There are elements that are similar.

Specifically, "simplicity" is my all-time favorite principle, I think, for any type of design. Whether it be game design or furniture or fashion, there's an elegance to simplicity and you don't have to delve very far to see dozens of different classic quotes: "Less is more." There's an Albert Einstein quote that I mention to game designers where he said, "Everything should be as simple as possible, but no simpler,” which is a really elegant way of saying that simplicity is a great thing, but warns you that you can overdo it.

All you have to do is go online you could find a dozen more quotes from various creative fields about how simplicity is really important. The humanity aspect of it, I think is something that we have found comes up quite a bit at Google. The principles of not being evil and helping people are talked about quite a bit within the company, and I think you can apply those almost anywhere. I don't know that VR is particularly special in that way, but I don't think you can go wrong, applying some of the same principles.

I’ve been struggling with the concept of VR and socialness. I'd like to get your input on this. I understand that it can connect people in a new way, but when something blocks out your immediate surroundings to that degree and it supplants actual reality, I wonder how social can VR really be?

I think specifically, you're right that there's an interesting paradox that we have with the current level of VR. We have technology that allows us to scan people's faces and facial expressions and it would be great to be able to project your facial expressions onto avatars, but the technology that allows people to enter virtual reality, at this point, involves wearing headsets that make it particularly difficult to actually scan their faces. I've seen a lot of creative solutions to approaching this problem, but nothing that I've found that's totally satisfactory yet. It's an area I'm sure we'll be delving into.

The core, though, of social VR is pretty fascinating in that -- at least with my own subjective experience -- being in a VR space and having other characters, even as fairly crude avatars, is pretty compelling, often in ways that surprise me. I think that gets back to the earlier research I mentioned of having other characters or people within your personal space. But I'm reluctant to jump to any conclusions and one of the things we find out with VR over and over again, just when we think we have a handle on what it's doing, it can surprise us and give us, sometimes a 180-degree reversal.

Reading your past writing, you've expressed how optimistic you are because things are changing so rapidly in video games. Obviously things haven't slowed down, so how is your optimism these days?

Pretty good. I guess I should say it's tempered with the realization that the industry is incredibly cyclic and as some things are going up, there are always other things that tend to be trending downwards, and we've had a lot of uptrends recently so that makes me a little nervous.

Regarding new things, VR and AR are obviously really interesting. eSports is something that's not technically new, but it's just been exploding. Korea has had a lot of that going for years, but the fact that it's gone everywhere. The fact that the spectator view of things has become such an important thing with what we see on YouTube and what we see on eSports broadcasts.

A lot of really exciting areas that are new and have plenty of unseen, unrealized potential. [There are] a lot of things that are evergreens, but I've seen the PC declared dead for gaming at least twice in my career and yet Steam managed to revive a lot of that. I wouldn't be surprised to see another cycle on the PC side within the next five or 10 years. It’s a very exciting time, very hard to predict, and that's what makes it fresh and interesting.