Virtual Insanity: How do you test in VR?

Virtual Reality (VR) is no longer just the domain of science fiction and imagination and has been gaining ground with developers, consumers, and technological giants alike. But releasing quality VR products is no small task, and requires careful consideration, planning, and, of course, testing. So how do you go about testing a Virtual Reality product?

With the development of our first VR project, the REDspace QA team had come up with a few approaches tailored to testing a VR experience in addition to our current testing standards and processes. Integration of any of these methods in conjunction with a good QA process will help ensure the release of a quality product and unique user experience.

Peer Testing (Spotter & Recorder)

Similar to Peer Programming, this method has two testers, a Spotter and a Recorder, working in tandem. The Spotter will stay “plugged” into the VR setup while the Recorder will remain on the sidelines, documenting anything as reported by the Spotter and observing interactions from the desktop running the VR application.

This method prevents the Spotter from having to remove the VR gear each time they wish to log an issue or take notes. Instead, they can simply inform the Recorder to take a note or log an issue as they reproduce it. It also helps with confirmation; a Spotter getting a second set of eyes on the desktop’s monitor can answer “Does this look right to you?”

Test Session Recording

Recording the test session with software such as OBS from the desktop can be very useful in capturing issues and bugs that can be referenced in tickets. Having the entire session recorded will allow tickets to reference certain timeframes to show the issue on the video. Videos should be denoted by the tester, day, and build. If time permits, the tester may be able to cut the video into appropriate sections and add those directly to the related tickets.

With the recording running in the background, developers can implement small testing functions. These could include having the screen flash a certain colour when the tester inputs a button combination, highlighting an area of the video to pay attention to when either cutting or viewing the video afterwards.

Developer/Debug Options

In game development, it is common practice to include developer options in video games (level select, infinite lives, developer console, etc) and VR is no exception. Including functions like the ability to see collision maps and other information output from the application, to take notes, or to bypass certain requirements can be invaluable to the quality of the application and the sanity of the tester.

Having a specific debug menu or ability to type out certain commands into a developer console will greatly speed up the time it would take to test specific issues/features, allowing the process of testing the application to become much faster than having to go through the regular user flow each time.

Other Options & Considerations

There are VR testing services available that can be implemented in a pinch, but it may be worth the time to search for local solutions as a way of drumming up attention for your application while getting real user input. There are VR Room establishments popping up in most cities. Reaching an agreement to test your application with real, local users can be a mutually beneficial solution by building a stronger presence within your city and giving a face to the product you are building.

It is assumed that the most basic way of testing these applications is with a single tester recording as they are able (pen & notepad, laptop outside of playing area, etc). Though this process is familiar to most testers, it is not the preferred solution, but rather a last resort. Automation in VR testing is in its infancy, but expect to see advancements as research continues in that area.

VR is an exciting arena, and now with consumer attention slowly seeking more and more applications within it, quality is a requirement for VR instead of a bonus.

We here at REDspace are thrilled to be able to put some of these QA testing methods into practice with our VR game, Gunball, on Steam for HTC Vive & Oculus Rift.