Beta Testing Your Game; Finding Success in Criticism

Recently we held the first in a series of beta tests for our upcoming game Final Contact. Or maybe we should have called it an alpha test since the game is still under development and we only tested a single level for its gameplay mechanics.

To give you a little background about Final Contact, it is a pseudo-3D space-based combat simulator created in a 2D development environment. It began as an on-again off-again project. I had been spending time developing a few low level casual games for HTML5-based web and mobile portals. Sometime last year when showing the prototype to a developer friend of mine who suggested that I should take this game to STEAM. I had added gamepad support for the game solely for personal testing purposes but, when he played the game with a gamepad, he quickly suggested that this game would be prime for STEAM platform, something that I had never considered prior. All well and fine but, it meant I had to redesign the game’s graphics at a higher DPI for the PC.

Scrreenshot from original prototype

So, earlier this year I went back to the drawing board and began to rebuild the game from the ground up and with that, I began an aggressive social marketing campaign. With a few gameplay videos and a lot of screenshots from the original prototypes, I began to promote the game and drew a lot of interest as well as some other developers who came onboard to help with the project. One of them was a very talented sound engineer from EA Games.

As things progressed I found myself at of development crossroads, so to speak. I decided this might be a good time for a beta test. Not having a full game developed for a proper beta and with plans to launch a Kickstarter, I decided that my beta test would be of four different levels, each one consisting of different game mechanics. Because the game is developed in HTML5, we were able to provide the game in a downloadable link that could be played on a web browser on any mobile device or on a PC with a gamepad, not requiring any installation.

Having a great social media start with a strong following of interest I began to solicit for beta testers. The first mistake we made was to start that solicitation almost a full two months prior to the actual beta. My goal was to get 100 beta testers and we acquired that amount in about two and a half weeks from the initial announcement. Unfortunately, with a large delay between sign up and the actual test, we drew far less response than we actually hoped. Although, one reason for this may be that many of those that took an interest are YouTubers and Twitch streamers. We know this because many of them emailed directly when we spoke with them through social media and they had expressed interest in streaming the game. However, we asked that no one stream the first level until after we completed the first series of beta tests with it. For the first test, which was of Level 1 of the game, the actual game mechanics were amped to be much more difficult than the actual first level would be. I wanted to see how far we could push an HTLM5-based game as well as test the core gameplay mechanics.

And so it began.

We threw the beta testers into it with no instruction, asking them to gauge how intuitive the game was and how well they were able to get a grasp on the mechanics of the game without any tutorial. Several days later, we followed up with a second email that included instructions and an outline on the game’s controls and objectives. I felt this was important because as a gamer, I am often turned off by games whose fundamental mechanics are difficult to grasp and I don’t enjoy having to go through a lengthy tutorials.

The Response

Being nervous that players might not like the game, we waited with anticipation until the first responses began to trickle in.

In the first round of testing, a number of users had issues getting their gamepads to work with their PC and we had not provided keyboard support as an alternative. This was a major oversight. As the second email with instruction went out a few days later, we added keyboard support and instruction for those who could not get a gamepad to work with their PC.

Out of our 100 beta testers, only just over 60 downloaded and played the game. And while they played it with frequency, only half gave a response to our survey form, but it was more than enough.

The response was phenomenal. Everyone liked the game. Along with a lot of positive comments, we also could see that the beta testers found the game to be intuitive and easy to grasp. A few commented that they thought the game was too difficult for a first level but we expected that, having made it that way for the first Beta.

Among the beta testers, there were some fellow game developers and friends from whom we gained important feedback about playability, which led to some adjustments in the acceleration and deceleration of the gameplay. Everyone loved the background music but more than a few found the sound effects to be a little too strong.

We also learned, having redesigned the game at a higher resolution for the Steam platform, that the download size was much larger than anticipated and that the final product for mobile versions would have to be substantially reworked.

Two weeks went by and because of the all talk on social media, we began to receive a lot of requests from other gamers interested in joining the beta. So we opened up to take more testers and received 150 more beta tester sign ups in 3 and half days. A couple professional game testers came onboard and one in particular gave some very insightful positive feedback. Especially about the HUD’s color, which was important to me. I had concerns about the HUD being all red but was hellbent on maintaining the color scheme. Earlier, the previous year, when I was developing the first prototype, I would frequently go out to a local pub to sit and test the game on my mobile device (and have a few beers), making a list of notes of changes to make and bugs to fix. Repeatedly, I was approached by strangers who saw me playing the game,  who asked me what the name of the game was and what platform it was on so they could purchase it. The red was so vibrant and eye catching, that I felt I had it right. After a lot of feedback, especially from the professional testers, I felt confident about the display color of the HUD.

We held a series of 4 tests, all in Level 1 as we kept making changes. Virtually all the feedback was positive, which seemed a little scary because I really expected more in the way of negative comments and actually hoped for it but, received little. One thing I’ve learned in the past is that nothing can get you on track like some truly negative constructive criticism.

There were a lot of positive constructive comments in the feedback. I definitely got a notable amount of feedback concerning gameplay on mobile devices versus the PC, in regards that the screen seemed a bit cluttered by the HUD on Mobile, which of course was understandable considering the addition of touch controls to play on a mobile device whereas the game was controlled by a gamepad on the PC. There was a small amount of negative feedback but, by and large it went ignored. Looking back, I believe this was mostly due to the fact that we receive so much positive feedback overall.

All in all, it seemed very successful first round of beta testing and we came away feeling pretty confident until a few weeks ago. I had been posting some screenshots of the final beta test on social media (specifically Twitter), when I received a return tweet with the message “I’m going to show this as an example of what not to do in a GUI“. Of course my pride was stung a bit after all the positive feedback but I wanted to keep an open mind so I reminded myself of that old poker adage, “Don’t fall in love with your hand”, and like any game developer passionate about his project, I was certainly in love with Final Contact and reminded myself about that. But, finally, there it was. The true criticism I had wanted from the beginning.

I took a look at the commenter’s profile on Twitter and saw that he was a GUI specialist and decided to reach out to him with a direct message. I told him we were still early in development and would certainly welcome any advice he might have. 2 days later, he sent me back one of my screenshots marked up with detailed changes and comments. I had actually been spending a good deal of time watching gameplay and promotional videos for some of the AAA 3D games like Elite Dangerous and Star Citizen. Like many game developers I often don’t have the time to actually play games. After giving it some hard consideration and in light of comparing it with other games, I realized he was right.

My targeting reticle in the center of the screen was too big and thick. My buttons did dominate the screen and I needed to find ways to free up screen space. I then went back and reviewed the small amount of negative comments we have received, realizing there was some valid points in there as well.

I had already been working on developing special effects of much higher quality than originally presented in the beta but with the new knowledge I gained, realized a lot more of it was going to be need a change.

Screenshot 1 from re-design on PC

Since then we’ve been working on completely redesigning the HUD, freeing up screen space, and creating a more high tech look. The new effects are quite spectacular (if I might say so) and we’ve replaced our 3D models with much more realistic, edgy enemy fighters, battle cruisers, and space stations. The entire look of the game has been dramatically enhanced by the changes and I can truly say that we couldn’t be happier with it at this stage.

Screenshot 2 from re-design on mobile

Since the day I first began working on Final Contact, it’s been in a constant state of evolution. Having a beta test and getting a lot of early work promoted on social media has really helped to contribute to that evolution. Looking back, I’m really glad that we had the first Beta and plan to do more in the near future. Getting feedback has really been crucial in the development process and I have no doubt that gamers, developers, and genre enthusiasts will all continue to contribute to this project as we get ready to take our game to IndieGoGo for funding and bring this game to market however; it was the truly negative comments followed by constructive criticism that snapped me out of my love affair with my game and really helped to clarify my perceptions about it, and ultimately lead to the most dramatic changes in the games design.

Dominick Gentile Lead Developer and CEO

www.final-contact.com