How John Carmack is bending Samsung's VR strategy

For the last year, John Carmack has been working on mobile VR at Oculus -- which is coming to reality in the form of Gear VR, a headset that Samsung is planning to launch. In concert with its upcoming Note 4 phone, it becomes a working VR headset. He told a rapt crowd at the Oculus Connect event in Los Angeles today, which Gamasutra is attending, about that work. It turns out that even the "most powerful" phone in the world, which Carmack says the Note 4 probably is, isn't perfectly suited to VR. He's been battling to get the company to give him concessions and make technological changes to how the phone operates when in the Gear VR headset. He's even had to hack the source code of the Android OS himself, at times, to make his point. But before we launch into that, there's reason to be optimistic that Gear VR will rapidly improve, says Carmack: "Samsung's technology ticks twice a year. They have big technology rollouts twice a year. We expect Gear VR to follow this path. We can roll out major new features as it goes on." "What I've learned from dealing with Samsung," says Carmack, is that "there's an icebreaker moment" -- "I push something through, and they do something, and it works really well based on what I suggested, and then the doors are open and we can get lots of things." He gave the example of increasing the refresh rate on the Gear VR during development. He was working, at that time, with its Galaxy S III phone. Android triple-buffers graphics, inducing a 48 millisecond delay into the system -- making VR impossible. Carmack pulled apart Android to hack that out. Though he'd written several emails to Samsung in attempts to convince them to give back that buffer, "It's easy to argue against an email, but it's much harder to make the argument when you can have the two things and stick them on your face and, 'Tell me this isn't better.'" He won that argument, finally. "After that, Samsung got me crazy things," Carmack says. At Samsung, "The software side believes in my judgment, and will go do useful things. The hardware side doesn't yet." The two "Achilles' heels" of mobile VR that Carmack wants to solve are positional tracking and the 60 Hz refresh rate of phone screens. His proposed solution for the refresh rate is to bring back interlacing -- CRT screens only draw half of the image on every refresh, line-by-line. We can do that with the OLED screens the Note 4 phone uses, too, Carmack says. With interlacing, "your brain will imagine it's a solid screen," says Carmack. "If we get that we can run a 60hz display with 120 fields per second," he says, the refresh rate will be much more reasonable for VR. According to Carmack, 90 Hz is the magic number where "95 to 99 percent" of people don't notice the refresh rate anymore. There is, however, "a strong reason to go beyond that, to 120 Hz," says Carmack: The math works out. "You can run 60 Hz content with double frames, 24 Hz content, or even 120 Hz." Interlacing also "opens up the possibility of dynamically changing that every frame based on the content we put in there" and improve the realism of what the screen shows, too. Overall, when it comes to interlacing, "I think it's doable without adding cost to the hardware" and "no additional bandwidth." "I'm betting it'll look at least as good as the 90 Hz" display of the Oculus DK2 dev kit, he says, too. "I'm trying to get that work done. I think it'll be pretty important." Achieving positional tracking will be a bigger challenge on mobile, he says. "The phones are powerful enough today we could run the same calculations" that the PC headsets do, he says, but the sensors in phones are not calibrated for VR and aren't necessarily hooked into the OS in the right way, either. He's working on ways, however, to "make that work on the mobile systems and power and graphics budgets" by cheating, to an extent -- there's a lot of push-back about cheating at Oculus, since the developers understand what the correct inputs are and what they want to track. But on the phone, it's just not possible yet. "Within the next couple months," he says, he should have a better solution. "One of the things that Samsung has done for us is giving us a path to get 120hz camera updates," he says, and he foresees the possibility of perhaps providing a "placemat" that offers the phone an absolute point of reference. "The reason this relationship is nice because we have a tight relationship with Samsung and one device," Carmack says. "Both in Oculus and in Samsung there were skeptics who didn't think it would work out," he says -- but as Gear VR improves, that skepticism is fading, on both sides. With improvements happening over time, well, Gear VR could really shape up to something that makes all skeptics into converts. For more from Carmack on the challenges of developing for Gear VR and Android, read our recent interview. For the announcements from Oculus Connect today, click here.