Good performance is very closely related to good user experience. User wants to see constant progress with the game, do the smooth gameplaying, graphics performance needs to be up to snuff and work across all different mobile devices (phones, tablets etc.) the very same.

Let's take a brief look on performance issues and what sort of things you as a developer can do to improve it. The easiest way to do this will be by approaching it through the testing, using whatever test framework but having that testing done on real mobile devices.

Mobile games require performance testing before those are published at markets. What type of performance testing is needed varies on the nature of game, graphics, gameplay, multiplayer functionalities and many other things. The most typical performance tests done for mobile games are general tests performance to determine how the game performs in terms of responsiveness, stability and refresh rate under certain workload or user interaction pattern.

The resource usage is important metric with mobile games as the power of CPU and GPU and the size of memory varies device by device. High-end devices are more capable to run anything whereas low-end devices can't even get the game up and running. This is not exceptional case that game is built for a high-end device, published and then gets complained by thousands of users that it can even get in the first loading state. Remember! Use test automation and you'll get your game's performance tuned to its best. Manually, you can't get any sort of understanding what happens underneath the UI/front-end/controls. Here is great example (with video) how to use advanced test automation techniques to get your game tested.

Different Types of Performance Tests

Performance test - this is the basic type of performance testing that provides information how quickly something is executed, how much it can be scaled (in terms of more graphics assets, or more connections/integrations) or how stabile the performance is in general. Typical metrics from performance tests are CPU load, memory consumption and frames per second (FPS). Based on this data game developers should further improve the game to either scale down certain functionalities, heavy graphics assets or something else to ensure game works on lower-end devices as well. With performance tests it is easy to fine-tune, further plan improvements on graphics, gameplay and connections, and to optimize game accordingly. One interesting 'benchmark' for measuring the user experience of your game is Gamebench, the least cheatable benchmark you can find out there now for mobile games, hardware and so on. Performance testing is very easy to be automated - find out more about it here.

Load testing - is somewhat the most simple form of performance testing. When certain load is exposed to the game and infrastructure the developer gets understanding of where possible bottlenecks are and what parts of game/infrastructure need optimization. Load testing can vary from basic server upload/download function to database connection and queries, and different types of inputs given to the game/infrastructure. During load testing developers typically get better understanding of concurrency issues related to the game and how those could be optimized. For example, if your game needs to pull something from back-end server and suddenly 300+ devices do that in the same WiFi network, you can get very invaluable information about your games's capability during this kind of scenario. Furthermore, load testing can be you to understand how game needs to 'compromise' when certain operations cannot be handled. Also, it typically gives great insights to hardware capabilities and how much real resources (CPU, Mem etc.) are needed for your game and what happens when those resources are exceeded. Stress test - determines if the system is able to handle everything, how it handles with data when the game infrastructure is overstressed, and how far can stress testing go before game stops responding and how are errors shown to users. Stress testing is important in case of games especially if you game requires connections to servers or needs to be connected to some other device. Typically not so many mobile games communicate everything instantly online but rather in parts. This limits the recurring use of networks, draining less battery and limits error scenarios as your game doesn't need to be send/receive something all the time. The stress testing helps you to understand what kinds of failures are the most valuable to plan for and what are the most common hardware failures or issues that you need to anticipate. 

One of the subcategories for stress testing is soak testing. It's also known as endurance testing and is used to understand if the game and infrastructure can sustain certain continuous load set on it. When soak testing is done, typically CPU, Memory as well as FPS and other important metrics are monitored to detect any potential leaks and issues. The goal of soak testing is to discover how the system behaves under sustained use. Some industry benchmarks can be seen as a soak tests, however, users should be aware of cheatable nature of those. The other type of stress test is called spike testing which is done by suddenly increasing the load generated by a very large number of users, data, inputs, or any other type of stimuli. This is not continuous but periodical and the thing is to find out if dramatic change in load will cause any errors or issues for the game and its infrastructure. The goal with spike testing is to determine if performance will suffer, the infrastructure will fail or how can those error states be handled in case of dramatic changes of inputs.  

Simple Steps to Start Doing Performance Testing

1) Write and conduct performance tests in whatever types of tests make the best use for your game. Make sure you stress the game itself but all infrastructure related to it as well. This means back-end servers, integrations with social media, and any other third party or external entity. All these things have an impact on performance of your game and through the performance to its user experience. You can even use Record-Playback approach to generate your simplistic performance tests! 2) Input everything to logs! This is very important and will help you to further investigate what happens in your game - even you would be monitoring it in real-time during the testing. Logs will maintain all that information and you can easily compare those later - even build-by-build - to see how things have regressed while building a new version of game. 3) Use only real mobile devices - and as many as possible. You'll get quickly understanding of how your game performs when you start doing basic smoke testing on hundreds of different devices. While running your game on those real devices, check at least these things:

Now - is you mobile game ready (and optimized) for hundreds of millions of users?