Accessibility Heuristics and Evaluation Criteria for Mobile Games

Introduction

Since the inception of the mobile app stores, the popularity of mobile games has grown substantially. With the rising popularity of mobile devices, the number of players with accessibility needs using those devices has risen as well. This warrants an understanding of the most up-to-date accessibility research specifically for mobile game developers to incorporate into their game production process. The worldwide population in 2021 using smartphone devices is 3.8 billion people, a 6.1% year-over-year growth (Newzoo, 2021). Mobile gaming market revenues in 2021 will account for 52% of the global gaming market at a $90.7 billion valuation (App Annie, 2020). This industry is projected to continue growing year-over-year (Knezovic, 2020) and its players are becoming increasingly more diverse. More than one in five (20.5%) players of casual video games have a physical, mental, or developmental disability; this compares to 15.1% of the American population overall who are disabled, according to the latest U.S. Census data (PopCap Games & Chauteau, 2008). Older adult audiences also experience general physical and sensory decline over time (Rienzo & Cubillos, 2020). Mobile-only players between the ages of 37-64 make up 42% of the overall demographic (Globalwebindex, 2020). With this in consideration, the concept of accessibility has in recent years become a rising cultural subject within the mobile gaming industry. It has become even more popular in the face of increased dependence on user acquisition and targeted marketing campaigns for generating future business growth. The accessibility advocacy topic is also coupled with ideas around raising awareness around diverse player needs, expanding usability requirements for players with disabilities, and improving the readability of digital products for the average consumer (Rehman et al., 2021). Accessible content also aims to make accommodations for various human conditions and impairments, for example, blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity, and cognitive limitations (Ballantyne et al., 2018). However, initiatives towards implementing a holistic approach for making mobile games more accessible are often dismissed by the argument of its marginal impact on business growth.

Current Research and Accessibility Guidelines

There are many formal and informal standardized guidelines used by developers and researchers worldwide for evaluating and assessing the effective accessibility of their games. These guidelines, or heuristics, are tools derived from institutions and organizations such as the World Wide Web Consortium (World Wide Web Consortium [W3C], 2018), Nielsen Norman Group (Nielsen, 2020), AbleGamers Charity (AbleGamers Charity, 2018), Game Accessibility Guidelines (Game Accessibility Guidelines, 2012), and the International Game Developers Association (Bierre et al., 2004). They are also adopted by private development companies for broadening their player-base and even governments for writing and passing non-discrimination policies or laws. Although these heuristics are primarily used and applied to the development of games on all platforms, they can be used to generate comparable and equally effective guidelines specifically for mobile games.

Many of the heuristic guidelines defined in the research of this paper have evolved from several years of iteration and continue to grow in scope. Through more than two decades of rapidly evolving technologies, the development of evaluation criteria for accessible mobile games gathers strong influences from several core industries including the web, consumer products, and information technology. A strong example would be the W3C’s Mobile Accessibility Guidelines which are derived from the Web Content Accessibility Guidelines (WCAG) 2.2 and which are now in their fifth iteration since 1998 (World Wide Web Consortium [W3C], 2021; World Wide Web Consortium [W3C], 2020). It was through these guidelines which established commonly led accessibility principles and standards of perceptibility, operability,understandability, and robustness in UX design. The principle of perceptibility states that information and user interface components must be presentable to users in ways they can perceive. This is one of the most misused in mobile gaming considering the small screen space of mobile devices. Guidelines under perceptibility include providing reasonable default font size for content and interaction controls, minimizing the set of presented information on screen to a certain degree, and maintaining specific contrast ratios between legible text and background (World Wide Web Consortium [W3C], 2020). The principle of operability states that interface components and navigation must be sufficiently operable by the user. This is the critical aspect to reaching effective player interactions with the game in hand. Guidelines under operability state that interactable icons such as widgets must be big enough and have enough distance from each other to ensure safe targeting by touch (W3C, 2020). Moreover, widgets must be easily reached on the screen considering the limited range of the thumb and the way the user may hold the device. The principle of understandability states that information provided by a mobile application and its operation of user interface must be understandable. Mobile games lose their engagement when the player cannot understand its content or how to control it. Guidelines under understandability include clearly indicating that interactable elements (such as widgets) are distinguishable from non-interactable elements, maintaining consistent layouts and iconography, and utilizing native-device meta-data for detection by screen readers (W3C, 2020). The principle of robustness states that the application’s content must be correctly applied across all devices the user attempts to access it with. This has direct implications on multi-platform functionality and testing of mobile games. These guidelines also suggest that developers work with emulator tools such as Apple’s Accessibility Inspector, which allows them to check the accessibility of widgets across all device formats (Dobosz, 2017). Developers need to verify robustness in the game and on every device model for which it is intended to be played. The admissible implementation of these principles may also be confirmed by local regulations and laws which exist to allow acceptable levels of accessibility for software.

Governments of the United States, Canada, and the European Union have adopted many of the success criteria for accessible software under law using the WCAG. Since the establishment of these laws, many private companies have also introduced initiatives and formed proprietary accessibility guidelines in the development of mobile technologies. Many companies have published their guidelines in an attempt to drive development forward showing unique accessibility solutions. One example is IBM’s Accessibility Handbook which contains three specially crafted checklists for designers, developers, and project managers working in information technology. These checklists include verifications of focus enablement, color and styles, content structure, clarity, and successful use of the device’s voice recognition or screen reader settings (IBM Design Accessibility Handbook, 2017). Checklists like these serve as a positive and proactive example for how game developers could move forward with making their own checklists for various crafts and teams associated with the game making process. Interestingly, there have also been attempts by technology developers, like Apple Inc., to increase accessibility and usability of applications using third-party software. By scanning applications through this tool, it enables hardware-based screen readers to operate on top of formerly non-operating applications (Zhang et al., 2021). This research has shown significant improvements in the performance of user interactions using mobile devices and could provide the industry with another constructive approach to developing tools with similar considerations.

Another example of a published industry-recognized set of accessibility guidelines is the Game Accessibility Guidelines (Game Accessibility Guidelines, 2012). These guidelines have been a collaborative effort between multiple studios, researchers, and academics to produce a developer friendly reference for ways to maximize accessibility in games. The guidelines are broken down into six categories which apply to each of the key disability groups: motor, cognitive, vision, hearing, speech, and general. Each category is divided into three difficulties which reflect the effort and effectiveness of each accessibility feature: basic, intermediate, and advanced. Basic features can be applied to a wide range of game mechanics and include allowing control remapping, toggling on or off haptic feedback, and adequate sizing of interaction controls. Intermediate features require more planning or resources and include support for multiple input devices, adjustability of game speed, and full interface customizability. Advanced features require complex adaptations and include full screen reader support, voice over support, and including players with disabilities as a part of the core testing group of game features.

Furthermore, private research facilities have made significant progress in the mobile Serious games genre by improve gaming experiences for the middle-aged and older adult population (Salvador-Ullauri et al., 2020). Serious games are a category of video games designed for educational purposes. Korea’s Advanced Institute of Science and Technology developed a specific Mobile Game Design Guide supported by a practical list of evaluation criteria using games like Homescapes, Brawl Stars, and Fruit Ninja with a diverse audience having visual, auditory, and cognitive impairments (Lee et al., 2021). These design guidelines include various general mobile game improvements in the areas of interface, game rules, and payment systems. For example, in the area of advertisement and payment, reward items that are provided for advertisements is much preferable to interrupting interstitial advertisements. Similarly, allowing advertisements to be easily closed or canceled reduces frustration with the choice of viewing an advertisement which the player is not particularly interested in viewing completely. They also include sections on increasing usability for players with audio-visual and motor impairments. For example, providing compatibility with supportive equipment such as touch pens or controllers. The method in establishing the guidelines used a direct approach with a representative sample group of the older adult population, a survey questionnaire, and speak-out-loud in-person playtesting. This method is a robust example of how developers could advance their own design guidelines with a user-centric approach.

Finally, with the wide-spanning attention around accessibility in the private and public sectors, several non-profit organizations and foundations have begun working closely with the games industry and developed their own guidelines using a constructive audience of dedicated players with disabilities. The AbleGamers Foundation has created a set of access and challenge design patterns taught under their Accessible Player Experience Design Certification (APX). The 12 access design patterns are guidelines to reach better understanding of content by as many players as possible, including those with disabilities. Several of the access design patterns such as improved precision, clear text, and clear channels are also relevant for mobile game accessibility. Improved precision, for example, explains that specific players with disabilities may need to adjust the precision of actions so that they can successfully target, move, or navigate in the game or its interfaces (AbleGamers Charity, 2018). The 12 challenge design patterns are guidelines for enabling consumption and use of content by these players as well. Several of the challenge design patterns such as helping hand, slow it down, and training ground may be highly applicable to improving mobile game accessibility. Training ground, for example, is a feature that allows players to practice in a variety of ways at their own pace so that they can gain the skills needed to succeed in the game (AbleGamers Charity, 2018). These access and challenge design patterns are rapidly being utilized by several large AAA and mobile gaming companies to implement accessible features for players. The AbleGamers Charity also offers a service for developers to test their games and receive feedback or recommendations from players with disabilities. This could potentially be an even more robust opportunity for mobile game developers to improve accessibility of their games by testing them directly with disabled players. A consolidated list of mobile accessibility guidelines specifically selected from each of the aforementioned resources may be found below.

Evaluation Criteria

The W3C’s Mobile Accessibility Guidelines, IBM’s Accessibility Handbook, Game Accessibility Guidelines, Korea’s Mobile Game Design Guide, and the APX Certification are all transformational and scientifically established guidelines which have been utilized by industry leaders. But how effective are they as heuristics in the evaluation of accessibility in mobile games? There has been expansive research evaluating the effectiveness of these heuristics. Significant peer-review and development has been made around accessible games for mobile devices. Such research has included the exploration and assessment of direct solutions related to specific disabilities such as visual impairments, hearing impairments, and cognitive impairments (Jaramillo-Alcázar & Luján-Mora, 2017; Jaramillo-Alcázar & Luján-Mora, 2018; Jaramillo-Alcázar et al., 2017). Each of these articles have utilized and applied accessibility guidelines to the game genre of Serious games on mobile platforms and found severe violations of guidelines and accommodating players with their respective disabilities. The discrepancies for players with visual impairments included lack of readable default text size, lack of customizable text sizing options, and options to reduce distracting visual effects (Jaramillo-Alcázar & Luján-Mora, 2017). The discrepancies for players with hearing impairments included lack of consistent audio-visual feedback, use of audio-only cues, or no indication of who is speaking in game dialogues (Jaramillo-Alcázar & Luján-Mora, 2018). The discrepancies for players with cognitive impairments included inability to save settings, not enough hints for accomplishing game goals, and lack of controls customization (Jaramillo-Alcázar et al., 2017). These cases show us that using guidelines can help us also in identifying issues with our current games. In addition to this evaluation research, several qualitative studies, with a conceded effort to legitimize certain industry-leading accessibility guidelines, have found most top-grossing mobile applications to be completely inaccessible (Ballantyne et al., 2018). Here, design heuristics have been applied to identify accessibility issues for players with motor, hearing, cognitive impairments, and those influenced by the consequences of physical and cognitive decline due to aging. The study simply utilizes the guidelines listed from the W3C, reports with an observational analysis of mobile applications used by people with disabilities, selects the relevant accessibility guidelines, and then creates a coherent and digestible list for mobile developers to utilize. This method of evaluation can be then translated into digestible improvements that can be integrated into the mobile game production process. Considering the limited resources game teams often have, it is important to sequentially prioritize the improvements according to severity and total estimated effort before assigning new content tasks (Rehman et al., 2021). Additionally, for scientific analysis of the results, it is essential to have the necessary A/B testing infrastructure in order to assess whether improvements have resolved any violations or how key performance indicators have been impacted. This approach can be a cyclical and iterative process to creating positive change in the user experience for players with disabilities or impairments. A summary and practical step-by-step approach to evaluation criteria with the cycle for iteration can be found below.

Conclusion

There have been decades of research to form several acknowledged accessibility standards for the use of software, web, information technology, and games. After all this time and effort, enough thorough qualitative and quantitative analysis of industrially standardized accessibility guidelines has taken place and spawned an important niche group of researchers and developers who strive for positive change in our industries. Much of this success is due to the growth of rapidly accelerating technologies and passionate researchers around accessibility. Meanwhile, it is still clear that players with visual, auditory, and motor disabilities could experience more improvements in their enjoyment of mobile games if developers applied well-researched methods and solutions. A considerate approach to the application of guidelines using mobile Serious games can show developers ways of providing gameplay enhancements for those who need it. Robust and user-centric approaches by Korea’s Advanced Institute of Science and Technology show developers how to increase validity in the application of accessibility heuristics with a user-centric approach. There are a multitude of resources attached to this thesis research to help developers improve their design, implementation, and testing of features for making more accessible mobile games. Furthermore, as the need for more cohesive guidance and practical processes in the growing mobile gaming industry increases, this research provides appropriate examples and a robust starting point for that increasing need to be fulfilled. Lastly, this research is intended as a source of advocacy for all players to have the ability to enjoy mobile games that are played and loved by millions every day.

References

AbleGamers Charity. (2018, February 13). Ablegamers accessible player experience. Accessible.Games. https://accessible.games/accessible-player-experiences/

App Annie. (2020, December 18). Global Market Share. Retrieved July 29, 2021, from https://www.appannie.com/en/

Ballantyne, M., Jha, A., Jacobsen, A., Hawker, J., & El-Glaly, Y. N. (2018, November 25). Study of accessibility guidelines of mobile applications [Research article]. Conference on Mobile and Ubiquitous Multimedia, Cairo, Egypt. https://doi.org/10.1145/3282894.3282921

Bierre, K., Hinn, M., Martin, T., Mcintosh, M., Snider, T., Stone, K., & Westin, T. (2004, June 29). IGDA accessibility in games: Motivations and approaches. G3ict. https://g3ict.org/publication/igda-accessibility-in-games-motivations-and-approaches

Dobosz, K. (2017). Designing mobile applications for visually impaired people. In J. Estrada (Ed.), Visually impaired: assistive technologies, challenges and coping strategies (pp. 103–126). Nova Science Publishers, Inc. https://www.researchgate.net/publication/321319458_Designing_Mobile_Applications_For_Visually_Impaired_People

Game Accessibility Guidelines. (2012). A straightforward reference for inclusive game design. http://gameaccessibilityguidelines.com

Globalwebindex. (2020). Gaming personas - an audience profile on the digital attitudes and behaviors of six gamer types[PDF]. globalwebindex.com. https://strivesponsorship.com/wp-content/uploads/2020/01/GlobalWebIndex-Gaming-Personas-Audience-Report-2020.pdf

IBM design accessibility handbook [PDF]. (2017). IBM Accessibility Handbook. Retrieved February 26, 2021, from https://accessibility-handbook.mybluemix.net/design/a11y-handbook/

Jaramillo-Alcázar, A., & Luján-Mora, S. (2017). Mobile serious games: an accessibility assessment for people with visual impairments. TEEM 2017: Proceedings of the 5th international conference on technological ecosystems for enhancing multiculturality, (66), 1–6. https://doi.org/10.1145/3144826.3145416

Jaramillo-Alcázar, A., & Luján-Mora, S. (2018). An approach to mobile serious games accessibility assessment for people with hearing impairments. In Proceedings of the international conference on information technology & systems (icits 2018) (pp. 552–562). Springer International Publishing. https://doi.org/10.1007/978-3-319-73450-7_52

Jaramillo-Alcázar, A., Luján-Mora, S., & Salvador-Ullauri, L. (2017). Accessibility assessment of mobile serious games for people with cognitive impairments. 2017 International conference on information systems and computer science. https://doi.org/10.1109/INCISCOS.2017.12

Knezovic, A. (2020, October 9). Mobile gaming statistics: 85+ statistics for 2020 | udonis. Udonis. https://www.blog.udonis.co/mobile-marketing/mobile-games/mobile-gaming-statistics

Lee, S., Oh, H., Shi, C.-K., & Doh, Y. (2021). Mobile game design guide to improve gaming experience for the middle-aged and older adult population: User-centered design approach. JMIR Serious Games, 9(2), e24449. https://doi.org/10.2196/24449

Newzoo. (2021, July 1). Newzoo global games market report 2021 | free version | newzoo [PDF]. https://newzoo.com/insights/trend-reports/newzoo-global-games-market-report-2021-free-version/?utm_campaign=GGMR 2021

Nielsen, J. (2020, November 15). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/

PopCap Games & Chauteau, G. (2008, June 11). Popcap games research. GamesIndustry.biz. Retrieved October 10, 2020, from https://www.gamesindustry.biz/articles/popcap-games-research-publisher-s-latest-survey-says-that-casual-games-are-big-with-disabled-people

Rehman, U., Abbasi, A., Shah, M., Idrees, A., Ilahi, H., & Hlavacs, H. (2021). Analyzing and prioritizing usability issues in games. In Lecture notes in computer science (pp. 110–118). Springer International Publishing. https://doi.org/10.1007/978-3-030-77277-2_9

Rienzo, A., & Cubillos, C. (2020). Playability and player experience in digital games for elderly: A systematic literature review. Sensors, 20(14), 3958. https://doi.org/10.3390/s20143958

Salvador-Ullauri, L., Acosta-Vargas, P., Gonzalez, M., & Luján-Mora, S. (2020). Combined method for evaluating accessibility in serious games. Applied Sciences, 10(18), 6324. Retrieved December 17, 2020, from https://doi.org/10.3390/app10186324

World Wide Web Consortium. (2018, June 5). Web content accessibility guidelines (WCAG) 2.1. Web Accessibility Initiative (WAI). https://www.w3.org/TR/WCAG21/

World Wide Web Consortium. (2020). Web content accessibility guidelines (WCAG) 2.2. Web Accessibility Initiative (WAI). Retrieved July 27, 2021, from https://www.w3.org/TR/WCAG22/

World Wide Web Consortium. (2021, May 14). Mobile accessibility at W3C. Web Accessibility Initiative (WAI). Retrieved July 27, 2021, from https://www.w3.org/WAI/standards-guidelines/mobile/

Zhang, X., de Greef, L., Swearngin, A., White, S., Murray, K., Yu, L., Shan, Q., Nichols, J., Wu, J., Fleizach, C., Everitt, A., & Bigham, J. P. (2021, May 8–13). Screen recognition: creating accessibility metadata for mobile applications from pixels [Research Article]. 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan. https://doi.org/10.1145/3411764.3445186