Today I got a chance to try Sony's Project Morpehus -- its VR headset for the PlayStation 4. I was able to play two demos: Castle and Street Luge. The first is a stand-up demo using Sony's PlayStation Move wand controllers in which you can attack a dummy using your fists, swords, and a crossbow; the second is what it sounds like, mostly notable because you actually lie down to luge. Both demos work well, and both are a bit different from what is arguably the best-known application of VR in the game space so far: CCP's EVE Valkyrie. The Morpehus headset's current iteration also works well: the visuals are clear, it's pretty comfortable, and it has good head tracking. After the demos I was able to speak to Sony engineer Anton Mikhailov, from the Magic Lab in PlayStation's R&D group, which also developed the Move, the controller tracking in the PlayStation 4 Dual Shock 4 controllers, and and the PS4 camera. He also designed the Castle demo and works with developers on bringing their games to Morpheus.

Now that Oculus has been acquired by Facebook there's been talk of broader applications than games for the technology. Is there any difference in making a headset that's primarily for games and one that's not just for games?

Anton Mikhailov: That's a really good question. Traditionally, there have been headsets that are very much not for games. For example, the HMZ viewer from Sony, actually, is what they call a personal display or something like that -- a big TV experience. If you can't fit a big TV in your room, you get that. That's one kind of non-game experience, but Morpheus is not aiming in that direction. For the kind of non-game experiences we've talked about in the past, and I think Oculus has talked about as well, like virtual tourism or telepresence or that kind of thing, I don't think there's that much difference between that and a gaming-grade headset -- because you are really going for the feeling of presence and immersion. Both in gaming and in non-gaming, you still want that as a fundamental feature. In both cases I think the headset has to be good enough to deliver that.

Are you in the R&D group working directly with developers who are creating content for the headsets?

AM: Yeah. Because it's still an R&D project to some extent, we are basically talking to the developers face-to-face very often -- talking about the new prototypes, their learnings, their feedback about the prototypes and things like that.

Is the feedback from those developers going into the iterations of the headset as you're working on them?

AM: Yeah. The headset you're seeing now is version four or five, depending on how you count. Before that, we had prototypes. For every version we built... This is the version we announced at GDC, for the broad developer announce, but we had internal versions for first party that we iterated on and things like that. All of the feedback, we coalesce and then we try to roll it in as soon as possible into the next version.

Is your group under any pressure to deliver? I know there's no stated release date beyond "not this year." Is there pressure to release or is it just a matter of getting it right?

AM: Pressure internally? Externally?

From Sony.

AM: Ah, okay. I think the priority is to get it right, definitely. I mean, PlayStation 4 is doing really well... There's obviously pressure to deliver it well and deliver it soon -- I think everybody just wants to get their hands on it. But no, it's not as if we're just trying to rush it out to get some publicity for it. No, we're trying to build a good VR headset that's sustainable, that's good for a longer term, it's not just a temporary boost kinda thing.

Oculus obviously has been very vocal about what they think needs to happen with VR. Valve has been working on VR technology and talked a lot about it. Do you have anything to say to the VR developer community that you haven't heard said by these other companies?

AM: One thing that has surprised me about the Oculus side of things is that they're very locked into the seated experience. That's a surprisingly safe way to go. We're still very much experimenting with very different types of content. And people have done stuff like that -- like the flying bird demo that people have done with the Oculus, I forget which group did it -- but here we have the Luge, which is a lay-down experience, we have Castle, which is a stand-up, hands-tracking experience. So I think it's worth mentioning that it's worth experimenting with every kind of experience, no matter how impractical it seems, because even if you don't end up shipping that, you still end up learning nuggets from it. So it's good to experiment. So that's maybe one difference I've seen, fundamentally, from us. We're still very much trying new things and although, yeah, seated is very much a safe bet, and I think everybody agrees that can be done, and that it's feasible to do as a product. That doesn't mean you can't experiment with other, different things.

Now, these tech demos [Castle, Street Luge] were created by Sony. The people who make the tech demos, how do you collaborate?

AM: Well, the tech demo, the Castle, I wrote it. [laughs]

So that's not even collaborating. You're developing it.

AM: Me, Jeff, Rico, we worked on it together. I kind of did the game design for it, other people did the engine. So internally the Castle is operating on our own R&D engine. We wrote the engine from scratch for VR so we could kinda figure out what's the lowest latency, highest frame rate kind of experience we can get. Once we do that ourselves, it's easier for us to talk to developers kinda first-hand and say, "Here's how we did it." It may not be the prettiest looking demo, but we can say, "Here's how we get the frame rate up, here's how we get the latency down." Things like that. So that's what we do: We make demos ourselves, to the best of our ability, and we go to developers and say, "Check out our demo. How do we take the best parts of our demo and roll it into your beautiful-looking game?"

One thing I was really struck by that Palmer Luckey said last year, is that more than good graphics, what's going to be important is really high frame rate, low latency.

AM: Right.

And in the end that's what is going to sell the experience and make people feel comfortable with it.

AM: I think so too. It depends on who you talk to -- there's definitely a graphical limit that you can't go lower than. Certain people say, "I don't feel presence" or "it's too much of a degradation from a typical gaming experience." But yeah, definitely framerate and latency should have really high priority, much more so than -- now when you make a game, you can sort of say, "Oh, I'm going to choose to do a 30 fps game because it's more cinematic," or whatever, or, "I'm going to cram more graphics into it." In VR, it's not really a choice. You have to go at least 60, preferably higher. It's kind of a new bar. There are new rules you should obey as a developer. Frame rate and latency are really, really important.

Do you think it's important for Sony as a company to make games -- I mean products, not demos -- that illustrate what this is for?

AM: Yeah. Our first party studios are very much on board for creating unique content. Sony's a big company. Obviously we have movie studios, music studios. There's a lot of potential for collaboration. Personally I think we're just as excited for other people making things for this device as we are our internal studios. When we talk to our developers we have an open forum between first party, third party, indie developers -- everybody. So everybody's kind of in the same boat, because we're all trying to learn what works in the space. We don't have a hard split between, "Here's the Sony studios doing the secret Sony stuff, and it's going to be the best of the best," or whatever. No. The third parties share with the first parties, the first parties share with us, and the indie developers are always welcome to join. So far it's been very collaborative, and a good process.

I don't know if you can speak to this, but how much of the rest of the Sony PlayStation organization, the developer relations and the publishing stuff, have been able to take this out to the wild and get developers interested or give them guidelines and help, and all that kind of stuff?

AM: Oh, it's been great. We've talked to third parties. I've personally done something like a hundred briefings of just third party people, indies, big studios, little studios, medium studios -- we actually have allotments dedicated to indie content, triple-A content, so we're trying to cover all the bases. We're not favoring anybody. We're favoring people who are excited about VR and want to make a product. That's really the only guideline -- it doesn't matter if you're internal, external, big or small. We're just trying to get the best possible VR content out there.

VR came up suddenly, progressed rapidly, but I don't feel like we know where it's going or how it's going to do.

AM: I've been at Sony for awhile now. This is really the first time where essentially... I don't want to say "everybody" because I'm an engineer, and to me "everybody" means 100 percent, but 99.9 percent of the people have come away with a huge grin on their face. You don't see many products that just have that high of a hit rate. I'm not in the business of predicting the future, but all signs are very positive toward this technology and this gaming genre as a whole.

I really also hope that we see a lot of inventiveness. I think in a lot of ways that developers are not going to be able to rely on the tropes and conventions that they've come to rely on.

AM: A big one is that you just can't take control of the user's head. You have cutscenes in games. You have lots of kind of head bob, and animation, and filtering effects. You get these random effects on the player's vision, and camera effects. That stuff doesn't make sense in VR -- your eyes don't get lens flare. Your eyes don't get random stuff stuck to them, film grain noise. All of that stuff, often on the art side, they have to rethink it. They have to say, "Oh, that looks nice, but it doesn't work in VR." That's one of our jobs at R&D and dev support, is to educate developers and say, "Try and let go of some of these techniques," and it can be hard, because they've spent years developing them. So to tell them, "Hey, knock out all of this nice animation stuff because in VR it doesn't feel good." So you've got to educate people on that.

And essentially, in addition to working on demos and working on the hardware, you're also working on that.

AM: So the reason we make demos is so we can say, "Look. Here is an experience that we think works well. And take note of the things that are done well. Suggest things that we could do better." If a developer makes a game, and, say, people get motion sick in it, they can have a reference for content that doesn't get you motion sick. "What are you doing differently that's triggering that?" Because we know it's possible to make plenty of experiences that don't trigger any kind of nausea. But often developers have some kind of weird animation -- head bob is a common one -- they just leave it in and that makes people very sick. So it's good to point to demos and say here's a way this has been done. That's a big part of educating people.

As much as you can talk about it, do you think the hardware is going to change a lot?

AM: You mean for the final product? For the first final product?

Yeah.

AM: I really can't talk about it. There's definitely a lot of ideas. But what will be final is going to depend on lots of trade-offs and technology advancements. There are a lot of decisions to be made. We're not ready to announce any kind of final hardware specs. But in the future VR is going to evolve, right? This is version one. PlayStation, Sony, they're in this for the long haul. There is a larger road map.

Are you fighting for things like field of view [FOV] or resolution? Do you have any things you think are important to the hardware more than other things? What do you prioritize?

AM: It is a tricky balance between all of these things. Essentially, for a given resolution, let's take 1080p. It's a good sweet spot for developers: 1080p/60, it's a target they understand. If you go with 1080p/60, FOV over 100 [degrees], the pixel density starts to be quite low. So it starts to be hard to read text, stuff like that. So I feel for 1080p, [FOV of] 90, 100 is a good sweet spot. If the screen res goes up, you can start to have higher FOVs. So I think there is a balancing game there. We're trying to make the most balanced system we can.