Head-Tracking for an Abstract Reality

When you scuba-dive, a good snorkel mask makes all the difference in the world. Your eyes are your most finely tuned and vulnerable perceptual instrument, and the job of a mask is both to protect them and to provide an environment in which they can function. If water leaks in, if the masks fog up, if it presses too firmly into your head, you are no longer immersed in a magical underwater world. Instead you are uncomfortably aware of your land-lubber body as it falters in an environment it didn’t evolve for.

VR has been around for a while, but what Oculus has done is the equivalent of creating a better scuba mask: one that doesn’t constantly spray salt-water into your eyes. And in my opinion their most important innovations have been in head-tracking. This means the in-game camera gracefully matches the movement of your head. It’s a complex challenge, and that Oculus has finally gotten it right is the reason so many of us in the tech world are confident that VR is here to stay.

This isomorphic treatment of head movement makes perfect sense for a symbolic virtual reality – one in which the imagined self on the other side of the VR veil is a binocular animal in a universe with three spatial dimensions. But the same assumptions cannot be made for a truly abstract virtual reality like SoundSelf. In SoundSelf, the participant isn’t submerged like a scuba diver into a magical vision of of our own universe – they are instead invited to leave their body behind and forget that they’re human at all. The trouble with an isomorphic head-tracking treatment for SoundSelf is that it makes the participant aware of their body, aware of the visualization as a three dimensional “space” in which they “occupy.” And while it doesn’t totally break the experience, it’s like a bungee connecting the abstract perceptual flow to the memory of being human.

So the question remains: what do we do with head tracking?

By far the most common support inquiries for SoundSelf’s alpha surround our lack of support for head tracking. For better or worse, people expect head tracking in virtual reality experiences, abstract or not, and when head movements don’t provoke a virtual response they think something is broken. So ditching head tracking is not a serious option.

The most attractive technique we explored mapped geometry movement and color to head acceleration. Like rosary beads or whirling dervishes, repeated physical actions can deepen a trance, and I loved the idea of encouraging rhythmic head rocking. These experiments felt great while I tested it for about half an hour: there is a gentle intimacy to feeling the world spin and brighten as your head moves from side to side. But there was one disastrous flaw: upon taking off the VR headset and returning to reality, I felt dizzier than I’ve ever felt in my life. I couldn’t see straight or drive safely for about five hours. Disastrous.

The solution we finally settled on is to keep the visualization stationary in the center of vision, but wrap it in a sphere that *does* respond to head movements isomorphically. SoundSelf begins with the sphere entirely visible, but as the experience gets under way it slowly fades out, coming back into prominence again only when the participant moves their head. It communicates to the participant, “yes, I’m here, nothing is broken” without compromising the abstraction of the virtual space. Players have reported feeling the world respond to their head movement, but not being able to put their finger on what it is that gives that impression.

You’ll have to look closely at the video below to see it. It’s quite subtle outside of the head mounted display.

There’s no getting around the fact that SoundSelf’s implementation of head tracking is in place exclusively to acknowledge user expectations. Head tracking does not directly enhance the experience as it does for most virtual worlds. I know the question on the tip of many of your tongues is “why then call it VR?” The simple answer is that while SoundSelf works without a VR peripheral like the Oculus Rift, it achieves its unique goals with tremendously greater force when it fills the visual field. To me, what makes the Oculus Rift an attractive device for SoundSelf is that it offers a much higher bandwidth passage from image to mind – a distinction that is mostly cosmetic for experiences structured around goals or story, but core to an experience that is structured around sense-perception directly.