Steam reviews study suggests 'bad design,' not bugs, irks players most

A recent study published by the Empirical Software Engineering journal examines negative and positive reviews for over 6,224 games left on the Steam platform, and the information could provide useful data for developers looking to better understand player concerns.

Conducted by researchers at Queen's University in Canada, the academic study delves into several factors that come into play when a review is written on Steam, such as examining hours logged before the review was posted, whether the game was indie or not, and complaints of performance issues.

A particularly interesting conclusion the researchers came to was that players complained more about game design as opposed to bugs. As the table below shows, only 8 percent of all the reviews examined mentioned bugs in games, while 34 percent of the reviews mentioned the cons related to game design.

The study didn't define "game design," which seems to be used catch-all term for various elements of a game, but "bad design" player commentary included "Gameplay is so boring" and "Not particularly engaging," according to the study.

In negative reviews left by players, 17 percent mentioned the bugs, while 57 percent mentioned the cons related to "game design." As the study reports, the percentage of reported bugs in reviews is low compared to the cons of game design. This could suggest that players value good gameplay design over technical quality. 

In addition, 42 percent of reviews that mentioned bugs in a game were positive reviews, suggesting that having bugs in a game did not necessarily lead to negative reviews being written. The study found that negative reviews were often written when severe bugs (like preventing a player from progressing) were prominent. 

Be sure to check out the entire academic study here, which includes more findings from the research conducted on reviews.