A Conversation with Oculus VR Founder Palmer Luckey

Palmer Luckey, founder of Oculus, is sure that he's got some tricks up his sleeve to perfect the Rift headset ahead of its official 2014 launch. At Gamescom, Luckey excitedly told Gamasutra that he has some ideas about improving his head mounted display -- though he didn't specifically divulge what they might be.

Luckey is well aware he might face competition soon, but he's also confident that his tech will keep his startup ahead of the curve. And following from his GDC Europe talk, he explains what it takes to make a game look and feel right in VR -- warning developers that they may have to unlearn genre conventions that have served them well for decades now. He also explains why he's excited by mobile as well as PC applications for the Rift. 

Things have really come along -- suddenly you've attracted an incredible CTO. What's the immediate future for Oculus? 

Palmer Luckey: The immediate future is to continue to working on the consumer version of the Oculus Rift and continue pushing toward that as fast as we can. The tracking, the optics, the display, as much as we can, before the consumer release.

As John Carmack is coming in as CTO, what is he concentrating on?

PL: He's really focusing on a lot of different things. He's working on improving our SDK, he's doing a lot of the work around mobile right now, and optimizing and seeing what we can build on mobile.

And part of it is when you optimize for mobile, if you can get it to run on low-spec mobile hardware, then it's easy to get it to run on low-spec PC hardware. So optimizing our SDK and demos around that helps on PC and on mobile.

But really, he is an incredibly smart guy. He is a wizard of hardware, software, graphical pipelines and everything in between. So he's really instrumental in a lot of the stuff that we're working on.

Have you said exactly when the consumer release is going to be, or is it more of a ballpark?

PL: We haven't. Sometime next year is a good ballpark. We've been telling people it'll be multiple months, not multiple years. The thing is, we don't want ship something that's not finished, or tie ourselves to a deadline and then either, (A) push things to make a deadline and then ship an inferior product, or (B) have everyone say, "Oh, they're late" when really we just want to make sure we do everything right.

I know you've been working pretty closely with some software developers. I've played EVE Valkyrie.

PL: It's awesome.

How do you feel about having software ready? I know you don't have direct control over it because you're not making games yourselves.

PL: I think we're going to have a lot of software ready, because there are a lot of game developers from indies to triple-As that we've been talking to -- some of them publicly, some of them are working behind the scenes -- we're not worried about content at all. There are so many people that are working on VR games. We've already seen a lot of really cool indie demos. But even a lot of triple-A developers are excited about the possibilities of VR and porting their games over.

One of the most interesting things about your GDC Europe talk is that you said that while in the beginning, we'll probably see a lot of FPS games because it maps well as an interaction paradigm --

PL: I hate the word "paradigm!"

It makes sense, though, in this context.

PL: Yeah, it does. The one time I've used it and it made sense.

It does make sense in terms of what you're saying -- but you also said you really hope to encourage a lot of different types of experiences.

PL: First person shooters are the obvious choice, because everyone's made them, there are lots of people playing first person shooters. But even games that are first person but are not shooters, like EVE Valkyrie -- its gameplay is very distinctive compared to most first person shooters, and it's a really, really good experience. Or we're showing iRacing downstairs, and that's a racing simulation title, and that's fantastic in the Rift.

And then you have all these crazy things -- you have people making 2D side-scrollers that wrap around you, or third person games where you're controlling multiple characters by looking at them -- just all kinds of really crazy stuff.

Obviously you got a really good head start, and you've got really good technical chops. But in the end, this is probably going to be one of the first products in this sphere, but there are going to be people following you. What is going to keep you ahead of the curve as competition starts to catch up?

PL: Well, we have the first mover advantage. We've got a really good SDK that makes it really easy for game developers to port games over and make virtual reality games. I think we've got the best team in the industry so far. I think that we're going to continue to make the best hardware in the industry. As long as we can make the best hardware, have the best people, and make the best software for developers, I think that we'll be able to stay in front.

And there will be other people, I'm sure, and in a way that's good, because the more people get into the space the more that legitimizes it. We're not just one crazy company. It shows that there actually is a market and that there are plenty of people that are interested in doing it besides us, and I think that it might mean more content. The more players that there are in this industry, the easier that it will be to make virtual reality content if there are multiple platforms that it can be on.

Large consumer companies obviously have similar technologies, but probably not assembled. We don't know who, but you will probably have followers. Is it your internal tech development that differentiates you?

PL: I think the things that would differentiate ourselves -- if a larger player got involved and said, "Hey, we're going to make a really big push around, this" -- one, we really do have an awesome team for making a software development kit. I don't think they're going to have a very easy time making something something that's as good as what we've made so far.

And I think, hardware-wise, we have so much momentum and we have so many connections in that industry that unless this company is willing to take a huge push that's it's going to have a hard time making something better than what we're going to be pushing to the consumer market.

And like I said earlier, if one of these companies does decide that they're going to spend all of this money and make a really big push, then it only ends up being good for us. Because if there's more people in the market, then more developers can justify making content for VR. Because that's what's going to make our platform, is content that is made for virtual reality. And if there are 10 VR headsets that are out there that all work with the same games, then it's a lot easier for developers to justify making VR-only games.

You mentioned the fact that there is a market. But do you have any sense of how large or what the potential is for that?

PL: It's really hard to estimate. We can sort of guess that there are a lot of people, but we can't really put a number on it because there's never really been a VR headset that really had any kind of shot at consumer success or has gotten as much attention as we have.

I think that a given is probably at least a couple hundred thousand people globally, because that's just a small fraction of the hardcore PC gaming market. So even if that's the only market we tap into, hardcore PC gamers, then we end up with a sizable number of people.

The real question is, will we be able to get less hardcore gamers, work on lower-end hardware, or even work on mobile hardware? Will we be able to expand that to where there's a lot more people?

In the end, it's just going to be a USB device -- or at least that's what I think. So have you considered working with consoles?

PL: There's no technical reason we cannot work on consoles. It is not up to us. It's up to them.

You have to be certified.

PL: It's totally up to them. That's really the only issue. You can make a game for a next-generation console that looked up to a Rift and played it, but it's up to the console manufacturers to allow you to have a USB device that passes the data over.

Is that something you're pursuing, or are you more worried about focusing on your first hurdle?

PL: If we were talking to the console manufacturers, I wouldn't be able to tell you if we were, so I can't really comment on any of that. But right now we're pretty much focused on PC and mobile, because they're both open platforms that anyone can target.

It's interesting that you have a mobile focus. Can you talk about that a little bit?

PL: Sure. When I say "mobile," I don't mean mobile games as in Words With Friends or Candy Crush, or whatever it is that people play on these devices. Not to insult those games, but that's not what we're excited about. We're not excited about these casual mobile games that people think of when they think of mobile gaming.

The reality now is that PCs, while they're getting more powerful, the mobile market, that's where things are really moving. They're getting much more powerful very quickly. If you look at a phone from three years ago compared to now, it's something like 10 times more powerful in that tiny little space. And we think that that's going to continue over time.

Mobile, it's an open industry. On Android, anyone can make an application. There are a lot of cool things you can do right now with the relatively weak mobile hardware we have today. You can make a lot of cool experiences.

One of the things we've been showing is a VR cinema, a virtual movie theater. And we can run that off of a phone. We can plug a Rift into a phone and you can have a huge screen in a virtual movie theater. You can imagine going on a plane or a train, or whatever, and being able to run that off of a mobile device.

But where it's really going to be interesting is when phones get as powerful as the PCs and game consoles of today, and are able to run really rich experiences. And potentially you can even have these mobile chipsets integrated directly into the headset. You can have a complete untethered --

Like a Tegra or something.

PL: Yes, exactly. Put something like that in a virtual reality headset. It wouldn't need anything else to run it. That's what we're really excited about.

Palmer Luckey

It's going to be interesting to see just how uptake is from consumers. In a sense, it's unpredictable. Have you done real research into it?

PL: We're marking the most badass product we can. If it's good enough we think enough people will buy us to keep us in business.

It's a real challenge to produce consumer products. Have you made any announcements? Are we talking about retail or online?

PL: It's really too early to say. Like I said, we're still working on the hardware. We're starting to look into that stuff and figure out whether we're going to go retail or direct, or specific partners. We don't really know yet. We need to figure out what exactly we're shipping, what it'll cost, and kind of how the business side is going to work, before we do that.

That's an important consideration, the business side. As you're saying, cost is going to determine a lot.

PL: We want to make it cheap as possible. What we've been saying is that we want to keep it in the same price range as the dev kit, so somewhere around $300, but really we'd like to make it even more cheap than that. One of the reasons that the consoles are able to move so many units is because they sell them with the expectation that they're going to make back the money later on games and stuff, off their marketplaces, rather than actually making it off the console. They're even taking a loss.

So who knows? We don't want to make it so that people are tied into anything to use the Rift. You should be able to just buy a Rift. But it would be really interesting -- let's say we're able to sell a bunch of Rifts, and we say, "Hey, the average user is buying this accessory, that accessory, and this game, and we can make this much money," or if we were to work on first-party content and make back a few dollars that way. Potentially, you could sell the headset even cheaper than $300. If you can get the price really low, you can make it an impulse purchase.

The way that they make back the money on the consoles is charging the publishers, in general. With direct digital, it's a rev share, with retail, it's manufacturing costs and stuff. You don't have exactly the same model in mind.

PL: Exactly. What I'm saying is we don't want to mimic that model. It was just an example of how, being able to bring those costs down, they're able to move so many units. If they were selling you an Xbox for what it actually costs to make on day one, then a lot fewer people would be purchasing them.

So essentially you're saying, without being specific, you want to innovate around the business and not just rely on selling hardware to make back your money.

PL: We don't want to sell things at a loss. We want to make sure we can make money if we only sell hardware. At the same time, we can have a lot thinner margins if we can find other ways to make money. People have been speculating -- "Oh, what does that mean? Does it mean that there's going to be a subscription? Does it mean that you're only going to be able to get it if you buy it with my Comcast internet, or something?" But really, we don't know. We're looking at all different options. But we're not going to do anything that locks out certain markets, or anything, or requires you to pay a subscription.

You spoke in your talk about how you're hoping to help solve issues around creating VR, best practices and stuff. Is it going to be in the form of best practices, or are we going to see SDKs that help with judder, and things like that? Or a mixture?

PL: It's a combination. What we're trying to do is take as much of the hard stuff as we can -- we're leaving raw access, for everyone who wants it. People can get the raw data from our sensor and try to do sensor fusion themselves. We want to leave that open.

But for the vast majority of developers, we're trying to take all of the hard stuff and put it into the SDK and do it ourselves. So things like judder compensation, reducing motion blur, high tracking precision, doing positional tracking, inverse kinematic models -- we really want to make it as easy as possible for developers who don't understand all of the technical side of that to make games for VR. So there will be a lot of stuff in the SDK. So as we make our SDK better, it will be easy for them to make good experiences.

But some things we can't really put in the SDK. If someone wants to make a fighter jet game where this person just spins and barrel rolls continuously, that is probably going to make people sick, and that can only be solved by a best practices guide where we say, "Hey, that is not the best idea to do in virtual reality."

Or so many first person shooters have this interaction where you go into something, you run into it, you start shooting at it, and you start running backwards until all of your bullets are gone, and you hope that you've killed it before you run out of bullets. That's probably the most common interaction in first person shooters.

In VR, and actually in real life, running backwards is not a really very comfortable feeling, especially if you're strafing side to side as you run backwards, and especially if you're moving at 20 miles per hour, like you do in so many FPS games. That's something that can only be helped by a best practices guide and people figuring out what works in VR and what doesn't.

It sounds like you're really concerned not with just shipping the product but getting people to understand how they should approach the medium, so to speak.

PL: Exactly. Because it's so different from making games for PC. A lot of the things do apply, but so many PC games, if you do just take them and port them over to VR, there are a lot of things just innately wrong with those games in virtual reality that can't be fixed just by tweaking the SDK or reducing judder. They're things that are fundamental to the nature of the game.

One thing you spoke about during your GDC talk is that the reduction of latency is a huge priority on your end -- you want to get as much latency out of the loop as you can on the hardware side, right?

PL: That is absolutely true. And actually, the stuff that we're doing in the lab right now, we think that we've got latency basically solved. We think that, for the consumer launch, we're going to be able to get latency to the point where it's not even an issue -- it's a completely nonexistent issue, completely beyond the level of human perception.

So it is a really hard thing to solve. We think, on our side, we're going to be able to get the latency down to next to nothing. Where the difficulty is going to remain is with game developers, and how they do buffering in their engines, how they do vsync. How their game engines handle rendering and whether they can stay at 60 or 90 or 120 frames per second. And that's going to be the difficulty. Because if we make perfect hardware, developers still have to make low-latency game loops.

I got the sense from the talk that you feel very strongly that vsync needs to be there, and that frame rates need to be as high as possible, and that frame rates are more important than complexity of geometry.

PL: That is absolutely true. I think that it's really important in VR to keep in mind that sometimes you do have to sacrifice fidelity for framerate. Because you do want to have vsync, because if you don't have vsync then you have tears in the world. So you'll actually have objects that appear like they're being sheared apart or that they're actually shifted relative to each other. And that takes you out of the game very quickly, as that's constantly happening as you look around. So you do need vsync. 

It's worth mentioning that a lot of the hate out there for vsync -- like a lot of people go, "Oh, you should turn it off, because it adds latency in games," [but] when vsync is done correctly, it doesn't necessarily add a ton of latency, or a perceptible amount of latency, in VR. But there are also a lot of games that do a very poor job of vsync. There are even a lot of games where you want to turn vsync off in their control panel and then force it using the Nvidia or AMD control panel because it does a better job through there.

Good vsync -- yes, you really need to do it, or the whole world appears like it's tearing. And you really need at least 60 frames a second. Right now we're saying 60 because that's what our hardware is capable of running, but if we had a display that could run at 90 frames a second, it makes a huge difference. Enough where you're going to want to run at 90 frames a second as often as you can.

You already went from 800p to 1080p for the dev kits. How far do you want to push upgrades and improve the tech?

PL: 8k per eye. [laughs] I mean, that's just a number where you could roughly, approximately stop seeing pixels at the current field of view. Realistically, we're not targeting any specific resolution as "this is the right resolution" because until we get to that 8k by 8k or higher resolution, we just want it to be as high as possible. We're at 1080p in the prototypes that we're showing, but we'd like to push it even beyond that.

What most surprised me about playing it was the FOV, actually. Yes, I could see the pixels, but that was maybe less unrealistic, because I'm used to pixels. But the FOV stopping was a bit odd.

PL: The HD prototypes that we're showing do have a lower field of view than the dev kit, because we're using the same optics that we used for the dev kit in these prototypes. We just swapped out a panel, we didn't change the optics, or the ergonomics, or anything else.

The field of view for the consumer version, we do plan on increasing. And not just the field of view, but also the clarity of the optics and their sensitivity to adjustment, so that people can have a much more clear view across their entire field of view, rather than having it blurred in the edges.

For me, in some way -- and I'm sure you've done much more research, and this is just me saying this -- but the one thing I'd expect to create a sense of reality is peripheral vision, and feeling it wrap around you.

PL: I totally agree. It's a lot of different trade-offs. Optically, as you go past 100 degrees, there are a lot of limits of optics you run into, and it can't just be solved with clever design. They're just the hard limits of refractive optics, and it's very hard to get around those. You can greatly increase the size -- like, if you double the size of the panel, then you can get a little further, but you're not doubling the field of view for doubling the size of the panel. It's diminishing returns. You end up with a huge headset with a slightly improved field of view. There are a few tricks that I am trying that I think that are going to be able to pump the field of view up beyond even where we are right now.

One of the issues with going at a larger field of view -- we're already at a fairly low resolution in terms of pixels per degree -- most of our vision is focused out here [gestures to the sides of his field of view in real life]. Let's say that you wanted to up the field of view to 200 degrees. Not are you cutting the resolution in half, it's actually even worse than that, because you have resolution here that's cut in half, but you're also throwing away all that resolution into the edges where you can't, unfortunately, utilize most of it most of the time. So it's a set of tradeoffs. How much field of view do you have, and what kind of pixels per degree compromise are you trying to make? But, like I said, I have a few tricks.

You're making hardware, which is obviously a lot more challenging, in a certain sense, than making software -- at least in the sense that you do have to finish it, send it off to a factory to have it manufactured and put into boxes, probably in Asia.

PL: And it takes a lot of time to do all of that. So software, you can update till the day you release, and then the day after that. You can't do that with hardware.

These days, you can actually do deals for manufacturing that are a lot more agile than what was possible even five years ago. But how is that working for you?

PL: It's working well. It's just one of the realities of hardware, that you have to lock months in advance so that you can start manufacturing the hardware, shipping it over, getting it in boxes, and potentially getting it on shelves. It just takes a very long time and it is much harder than software in that way.

I'm assuming you're working pretty closely with some of the developers. This is just me guessing-slash-extrapolating, but Valve and CCP, you seem to be talking to them pretty actively, right? So are you getting feedback from developers and bringing that very actively into your pathway, or is it more research internally?

PL: No, that's one of the reasons we shipped these dev kits. A lot of the developers who bought these dev kits give us feedback either privately or publicly, and we take all of that into account as we design this stuff. And Valve, especially, has been super helpful, and they've been awesome to work with.

And it's good to have people -- there are a lot of developers out there who want to make virtual reality better. Beyond just making their own games, they really do want the technology to advance. So they are really interested in giving their feedback and helping us as much as they can.

To go back to something you said earlier, it sounds like you're not just concerned with getting the hardcore PC crowd. The early adopters --

PL: They're the most receptive to new technology, I think.

But at the same time, it sounds like you do want to see indies making games. You want to see imaginative experiences that are outside of the realms of traditional video games, potentially.

PL: Well, one of the things that I touched on in my talk was that VR, in a lot of ways, takes experiences that would have, by necessity, been quote "hardcore gamer experiences" because of the skill that it took to manipulate the view and hit targets. It took a lot of practice to be a gamer capable of playing these hardcore games.

Virtual reality, in a lot of ways, because you have head tracking and potentially more intuitive interfaces and input devices, a lot of these games actually become playable by people who haven't played video games as much. So in a way, it's not that we're targeting hardcore and casual markets. We're just trying to make a wider range of experiences available to everyone.

I will say that looking at ships to lock onto them in EVE Valkyrie... I mean, I know how to lock onto ships in video games. But still, I got it in a second.

PL: It's so incredibly easy to do. Whereas if you took someone who doesn't play a ton of video games and have them using a keyboard to control the ship and then using a mouse to look around and try to lock targets, I think they'd have a much harder time.

When we're on the cusp of something, it's just so impossible to tell what's going to happen. Do you have any sense of what's going to happen, or do you think you're going to release it and be incredibly surprised?

PL: Well, if I thought it, I wouldn't be very surprised, I guess! But, like I said, we're really focusing on trying to make the very best product we can and ship it out there. And I really do think that if we make the best product, and if it's really good, if it's as good as we can make it, then we will be successful. And if we're not -- if we make the best thing that we can and we can't be successful -- then maybe VR really isn't all that great after all.

But I don't think we really need to know how it's going to be or waste our time trying to predict it. Because there are companies that do nothing but try and predict what's going to happen in the game industry, or what's going to happen in the movie industry, or in the tech industry in general. And you know what? They're wrong so often. It would be a waste of time to sit and try and predict how the game industry is going to work in a couple of years, or in a few years. All we can really do is do our part in trying to make this awesome.

In the '90s we had the sense that VR was around the corner. I'm sure you remember.

PL: Yep.

It was the thing that was going to be the next thing, and then it just completely -- well, not completely went away, but almost dwindled down to nothing. Do you think there's a lot of pent up expectation?

PL: I do, and actually I think that's what kind of killed VR in the '90s. That there was all of this expectation for people who had seen movies, played games, read science fiction novels, and they expected that virtual reality was going to be this huge thing that was imminently going to change the world. And the reality is that the technology wasn't even close to ready. So people who hadn't tried VR were more excited than the people who had, which is kind of an odd thing for a technology.

I think that a lot of that pent up demand for VR is helping us right now. Because unlike most gaming peripherals or gaming hardware that does things differently, they have to work hard to actually convince people why their thing is better. Like the Wii U, they're trying to convince people why using this GamePad is better than just normal gaming. I think with VR, people already know why it's better. They get it. They've wanted it for a long time, so you don't have to convince them.

On the other hand, they have very high expectations. They are expecting something that is like The Matrix. So you have to get as close to that as you can before you release. Because if you launch something that isn't as good as technically possible and then they say, "Hey, it isn't actually thatgreat," then you're going to have a hard time convincing them to give it a second shot.