Game Design in VR: Pushing Off from a New Frontier

VR is demanding we re-investigate from the ground up all things we have been assuming about UI/UX, and that we look deeper into cognitive science to address questions about how we perceive our environments and what fidelity even means in VR. We're moving the requirements of the systems we design out into a far more literal space in a way that breaks basically everything, while amping up the requirements for how we convince our brains to operate in that space.

Interface mitigation becomes no longer the problem of closing the perceptual distance between screen and user, but instead preserving presence. This is one of the fundamental mistakes that a lot of folks make re. VR--it is not immersive by default. What it gives you is immediate presence that we can just about instantly break because of our lack of knowledge on how to make things work in VR--how often do you actually feel that you are there in VR? Almost never, unless you provide distractions that bring us into the territory of flow and real immersion. E.g. add a perceived threat and just like the screen disappears in a game as the player is pulled in (mentally in)--the things reminding your brain that you're not really there in that VR space temporarily get replaced by a more urgent, primal message: survive.

This means that as game devs we have an interesting opportunity to explore the space with a bit more freedom. Once the initial charm of VR wears off there needs to be a fundamentally compelling experience underlying to motivate the wearer to continue (assuming we have moved past the nausea)—the fun of pivot tables in VR will only compel the average person so far. But it also leads to some new questions about the nature of our (game devs') work:

Unlike standard games today (with perhaps some weak exception in kinaesthetic systems like the Kinect), VR gives the player extension. Whereas a monitor confines you to a window into another world, VR extends you into that very world, which fundamentally alters some core design principles. The reason that simply playing Tetris on a screen in VR doesn’t "work" is that the game is not designed with extension in mind—it is still transmitted through that window. The challenge, then, is to build systems that facilitate extension and presence (or at the very least stop breaking it). Do you ride the blocks down yourself? Are you the block? I’m getting motion sick just thinking about it, but you get the idea. Key is that we’re not strictly mapping our existing content into VR, we’re creating whole new content that is VR-specific.

Early ideas (to varying degrees of success) are showing up in the form of mimicry. Approximating spaces we intuitively understand, such as exploring a world, gives us some affordances by which we can more easily feel extended into those spaces. I.e. they’re more like real spaces. But how do we push the state of the art? How do we create game spaces and experiences that offer extension in ways yet imagined?

Because, in part, of the lure of 1-1 translation, a current tendency is to develop FPSes in VR space. The reality is FPSes in VR are very far removed from the flat-screened 2D experience. Such games get a bit of freedom leveraging the physiological arousal that comes with being attacked and the survival instinct, plus environments that roughly map onto geometries we understand (or sufficiently constrain our environments to block difficulties, like being in a mech). But as anyone who has spent any time in these worlds soon experiences, the mapping is loose at best, breaking down the longer we spend in that space. It turns out that running around with a gun in a space requires a lot more perceptual markers than we are able to currently produce. So the “success” of FPSes in VR (and once more that’s assuming you’re not nauseated) comes from an overly simplistic 1-1 mapping of a threat that creates physiological arousal over a certain threshold and (sometimes) keeps our brains distracted from all that is wrong. As designers that also makes us lazy because we don’t have to pay attention to the hard details (to be fair, exploring a design space on certain assumptions of what you’re trying to solve vs. what you are not trying to solve is not what is lazy, it’s rather that doing so can lead us to neglect a sorely under-studied design space).

So the interesting question becomes if we’re not re-building Myst-clones or FPSes or mech-shooters in VR (which are all great things, don’t get me wrong!), what could we be doing? And what can we learn from what researchers are doing in areas like cognitive science that will help us create presence and flow in VR-space when our old tools fall short? They won’t all fall short, of course, but given that the signal noise generated by the brain in VR space does threaten to overwhelm classic design techniques, how do we evolve?

Some really quick ideas:

And so much more…

Finally, we have to be careful not to artificially limit ourselves by the current or near-current technology coming out in VR. If we successfully push games out into VR space, we need to avoid recreating solutions that were designed for a different, perhaps fictional space and instead think about how we really need to interact in this new space. Only once we’ve pushed past the availability bias can we then start to consider the tools that are tailored for humans in VR (and not some Hollywood-ized recreation), and leverage what is a delightfully exciting new design space for game developers that will challenge us to our core.

Happy frontiering.

[ A lot of the thoughts here come from my work in cognitive science over the last couple decades and the work I do in game development as well as recent conversations I've been having in the industry. Always happy to expand if anything is of particular interest here, please don't be shy to reach out.]