Feature: 'Measuring Responsiveness in Video Games'

Following up a previous article on programming responsiveness, Neversoft co-founder Mick West continues the discussion with a cunning 'how-to' about using a digital camera to track responsiveness - benchmarking games from GTA through Heavenly Sword along the way. Before presenting his pictorial guide on creating an easy and relatively cheap setup for measuring responsiveness, West explains that measuring response time allows developers to verify their own assumptions, identifying bugs adding to the response time, and to provide an objective reference to the claims of the testers regarding the "tightness" of the game. Perceptions of changes in small variables like response time can vary by individual, and being able to measure it objectively will allow you to see if it has actually changed, and by how much. Game developers also have to make the decision of whether to go with 60fps or 30fps. 60fps will generally have half the response time of 30fps, which can be a deciding factor (along with the smoother motion, which is visually more appealing on fast moving games). However for some games there are other factors that influence the response time. Having an accurate way of measuring the response time allows the developer to more accurately and objectively make a decision on if 60fps is necessary, or if they simply need to tighten up their 30fps game. Using his video camera setup, West tested the responsiveness in several Xbox 360 and PlayStation 3 games, posting his results and conclusions from those results: Games that run at 60fps all seem to have a response time to 4/60ths, and while 3/60ths is possible, 4/60ths is a very good response time. Some games running at 30fps have a response time of 8/60ths or 10/60ths (and some peak even higher). Genji shows us that a response time of 6/60ths is possible while running at 30 fps. 10/60ths can be too long, especially when combined with the processing delays in flat panel TVs which can push it up to 12/60ths. 1/5th of a second (200ms) is too long to wait for a gun to fire, and introduces annoying sluggishness when moving around or steering a car. Some games have an inconsistent response time. Heavenly Sword varies from 7 to 18. If the system is capable of 7, then all moves should start in 7. Developers should verify ALL their response times, as other factors, such as animation, might be creating lag in specific places. You can now read the guide on creating your own video camera setup to measure responsiveness in games, as well as Mick West's results from measuring responsiveness in several popular video games (no registration required, please feel free to link to this feature from other websites).