Raw Data devs share their missteps so you won't make them too

Survios’ Raw Data has been one of the standout successes in virtual reality game design. But at VRDC Fall 2017 in San Francisco today, a pair of Survios devs took the stage to talk about how development went wrong along the way -- and what they’ve learned from the experience.

“Today we’re going to tell you everything that went wrong with Raw Data,” said Survios design director Mike McTyre, alongside Survios CT Alex Silkin.

The game launched in Early Access last June, and went through what McTyre describes as seven major updates to date. While it’s available on Oculus Rift and the HTC Vive (with a PlayStation VR version coming soon), the game’s playerbase heavily skewed towards the HTC Vive -- to the tune of roughly 80 percent.

Here are a few interesting excerpts from their talk: During development, the team was initially going to keep the game’s map count very small -- maybe as few as just two, said Silkin, with some color variants.

But when the game got popular, the team thought to increase the scope and add more maps -- but that turned out to be kind of a waste, because at this point just 21 percent of players make it past the fifth mission.

First impressions matter -- most people play just the early bits of single-player, and not PvP or co-op

“First impressions matter,” said Silkin. “A lot of players just played the first few missions and got the impression ‘oh, this is a game wher eI just stand in a room and shoot.’”

He says this kind of hurt the game, because the team spent a lot of time and effort making more complicated maps later in the game -- maps that most players never see.

The number two most requested feature for the Raw Data devs to add, according to McTyre, was player-vs-player combat -- but even though the dev team put a bunch of effort, again, into making a PvP mode they could be proud of, McTyre says only 13 percent of players have ever actually played PvP.

“Out of all our players, 80 percent basically, have only ever played single-player,” said McTyre. “That’s an important takeaway for any other VR product, especially since co-op was such a selling point of the game.”

Despite that, only 16.5 percent of players have ever loaded up co-op play. And again, only 12 percent of players have ever played PvP.

“This is an important takeaway for devs: while VR is growing, and there’s a lot of users, creating a PvP-focused product, or trying to build a PvP-focused community, is going to be a challenge for you,” said McIntyre. “Even if you’re going to try it, I would encourage you to have some kind of single-player experience as well.”

All of these numbers come from the analytics data Survios has been pulling on Raw Data, which the pair said have been vital to ensuring they’re understanding how people play the game. For example, they were shocked to see that only 1 percent of players ever used the game’s defensive abilities, or that at one point 50 percent of players were failing the first mission.

“Half of them are VR enthusiasts,” said McTyre. “They were just people who loved VR, and that was the biggest takeaway for us...why we had to change things and nerf missions and that kind of thing.”

Also, “one of the big lessons we learned was about the frequency of our updates,” said McTyre. “Originally we were too ambitious….and we were gonna shoot for every two weeks.”

“That...really didn’t work out,” he admitted. The team switched gears to try for a once-per-month update schedule, but in the end they had to settle on a once-every-3-months update cadence so they could put significant effort into each update.

“We were in this Early Access period and we had all these users, and we wanted them to come back and experience the new content,” said McTyre. “But what we realized early on was if we did small updates -- just one map, just one feature -- it wasn’t bringing the users back. So we quickly realized that we neeed to have longer update periods and bigger updates to bring people back. And when we did that, we saw bigger spikes in players returning because the updates were meaningful.”

People play the same game different ways on different platforms/headsets

Fun fact: Most Vive Raw Data players (79 percent) prefer the game’s “sticky” control scheme over the “Hold” or “Toggle” schemes, whereas on the Rift, the majority (72 percent) prefer the “Hold” scheme.

“If you’re releasing your game on different platforms, they have different controllers,” said McTyre. “So don’t assume that just because one of your control schemes is the most popular on one platform, that it will be the most popular across all platforms.”

And while the game was initially intended to only have teleportation-based locomotion, the team's decision to add PvP meant they had to figure out how to get joystick-based locomotion into the game mid-development.

“We just couldn’t wrap our minds around making Raw Data PvP with teleportation,” said Silkin. It wouldn’t be fun if an opponent could just teleport away at will, so the team decided (despite heavy skepticism) to try using joystick-based movement.

“Ideally this is something you should plan for; we added it mid development cycle, so it broke a bunch of things,” said Silkin. Since they hadn’t scoped for joystick locomotion, the enemy AI could be easily broken by players just backpedaling, and had to be fixed.

(And despite the fact that a vocal group of players clamored for joystick locomotion, McTyre says only 20 percent of players actually turn it on.)

Incidentally, to do joystick locomotion on PSVR, which has no joystick, the team tried a “backpedal” button -- and it works well enough that the game will ship with it.

“Our biggest challenge to date was porting to PSVR, because we didn’t plan for it,” added McTyre. Silkin agreed and explained that because of that, there was a lot of reworking that had to be done.

“We had a lot of issue with performance, mostly; we’re CPU-bound,” said Silkin. “Sony is very strict about performance; they don’t want you to get their customers sick.”

“Other importants things about PlayStation, besides not having a joystick, is that there’s no grab buton,” said McTyre. “So we actually had to change up our control scheme quite a bit.”

“One nice advantage? There’s a lot of buttons!” McTyre added. “We were very happy to have a lot of buttons to play with.”

To optimize for the PS4 and PSVR, the Raw Data team switched over to a forward renderer and did some aggressive changes to the game’s levle of detail -- not only on assets, but on enemy animations (decreasing quality when they’re far away, or behind the player), enemy count, and other aspects of the game.

The game’s biggest cost was enemies moving around the world, “so we just kinda refactored our systems to not have as many things attached” to things in the world. After a lot of time and effort, they had something that could stand by itself on PSVR.

In closing, the pair offered fellow devs some tips for selling your VR game: