Music and Game Design at IGDA Switzerland Demo Night

The newly resurrected IGDA Switzerland Chapter hosted its first Demo Night in Zurich on Monday 4^th April around the theme of “music and game design.” With generous support from the Swiss Arts Council, Pro Helvetia, and SAE Institute Zurich, the auspicious event brought together a wide spectrum of perspectives voiced by the following line-up of industry veterans and newcomers:

MASAYA MATSUURA (NanaOn-Sha), the father of rhythm-action games and creator of the PlayStation One classics PaRappa the Rapper, and Vib Ribbon.

FERNANDO RAMALLO, (byfernando.com) who worked as co-designer, lead programmer and artist on Panoramical, which was nominated for the Excellence in Visual Art, Audio, and Nuovo awards at the Independent Games Festival (IGF) and showcased internationally at prestigious venues like the San Francisco MoMA.

Audio-visual artist, MATTHIAS MOOS (Mativa), and Zurich-based musician, GEORG BLEIKOLM (Knor), who collaborated on the first European transmedia app that combines virtual reality and music for, Don’t Go – The Purest Of All Sounds.

DAVID CANELA (david-canela.com) is the independent game and audio designer behind MODSORK, which features synchronized beats and music that adapts to gameplay. 

JANINA WOODS (Ateo) who is co-developing the virtual reality "rail music shooter" called, Shiny, which was a VR Jam 2013 Finalist and a winner of Pro Helvetia's Call for Projects: Swiss Games 14/15 contest.

The audience was asked to contemplate the elusive boundaries between music, gameplay and interaction. Does the gameplay drive music, or does music drive gameplay? Or is gameplay a concern at all?!

Matsuura-san presenting his latest game, furusoma, during a pre-recorded video presentation sponsored by SAE Institute Zurich.

furusoma, presented by Masaya Matsuura

Masaya Matsuura’s video presentation helped set the context for discussion due his long and influential career, which he traced back to the conservative music industry of the early 90s. Matsuura-san’s musical background and experiments in the burgeoning field of “multimedia” eventually found a perfect home on the PlayStation One—fuelled by the pioneering spirit of the era. PaRappa the Rapper (1996)—the first of his games to be released on the platform—became an instant classic due to innovative gameplay, quirky characters and music. UmJammer Lammy (1999), and Vib Ribbon (1999) further refined the rhythm-action formula in which music drives gameplay, thus paving the way for franchises like Guitar Hero by Harmonix, and SingStar by SCEE. Thankfully Vib Ribbon has been re-released as a downloadable PSOne Classic on the PlayStation Store, in case you missed it the first time around.

An original trailer for PaRappa the Rapper (1996), by NanaOn-Sha

Asked why music games tend to focus on rhythm and melody, Matsuura responded by stating that this certainly isn’t a steadfast convention, which he demonstrated with his up-coming iOS game called, furusoma (to be released mid-2016). Where games like PaRappa the Rapper, and Vib Ribbon use rhythm as the core game mechanic, furusoma challenges the player’s sensitivity to musical pitch, using a musical keyboard interface, through three levels of escalating difficulty to earn a highscore.

Panoramical (2015), by Fernando Ramallo and David Kanaga

Panoramical, presented by Fernando Ramallo

Matsuura-san’s approach to game design was contrasted by that of Fernando Ramallo who, together with David Kanaga, set out to create a player-driven audio-visual experience that eschews gameplay conventions like challenges and victory conditions. Players of Panoramical (2015) traverse audio-visual worlds, which they affect by adjusting various parameters. Each adjustment can have a dramatic influence on the experience on the game’s sounds and visuals—compelling players to explore the range of possibilities presented within each world to find their own balance between harmony and dissonance. In his Skype presentation, Fernando explained that this has, somewhat unexpectedly, resulted in players using the game as an instrument for performances in which players act as audio-visual DJs for a mesmerized audience. Without game rules, there is no right or wrong way of playing Panoramical because the music and visuals are treated as the audio-visual plaything.

Don’t Go – The Purest Of All Sounds (2016), by Matthias Moos and Georg Bleikolm

Don’t Go – The Purest Of All Sounds, presented by Matthias Moos and Georg Bleikolm

The presentation of Don’t Go – The Purest Of All Sounds (2016) by Matthias Moos and Georg Bleikolm shifted the boundary between music and gameplay yet further towards a purely experiential interactive experience. The iOS app is linked to Bleikolm’s music single, Don't Go, and his frustration with the present climate of xenophobia and rightwing politics around the world. The Zurich-based artists created an experience designed to disrupt the audience’s senses and immerse them in an otherworld without borders. Unlike Panoramical’s performative aspects, Don’t Go – The Purest Of All Sounds is a decidedly singular experience that presents music-generated visuals by Matthias Moos in virtual reality using Google’s Cardboard Viewer. Users simply rotate their head to explore the abstract audio-visuals, which can be seen momentarily in the above video for the music single of the same name.

The music single, Don't Go, by Georg Bleikolm, which inspired the collaboration with Matthias Moos.

Both Panoramical, and Don’t Go – The Purest Of All Sounds are exploring a new area of interactive entertainment which, Ramallo pointed out, requires educating the audience. This is especially true for players of mainstream video games who may come to the experience with certain pre-conceived expectations for gameplay. However, Radiohead’s decision to release their next album, Negative Space, as a virtual reality app for the Oculus Rift points towards a wider acceptance of music-based interactive experiences that—as stated by Masaya Matsuura—are set to become a genre of their own.

MODSORK, by David Canela

MODSORK, presented David Canela

The next presentation was by Swiss game and audio designer, David Canela, and his work-in-progress, MODSORK. The game sees players simultaneously controlling two on-screen buddies in the midst of arcade-style action. MODSORK creates a highly satisfying sense of feedback because each gameplay element is linked to a particular sound effect or musical loop. Canela describes the game as “beat-syncing,” in that game elements are synchronized to the beat—the game's musical rhythms. However, gameplay is not bound by the music’s rhythm—as in games like PaRappa the Rapper—but a sense for rhythm and the significance of sound effects helps players anticipate certain events. MODSORK’s gameplay and music are tightly interwoven and each play-through will result in different soundscapes being generated. This effect was illustrated by a photograph proudly taken by Canela at a recent gaming convention, in which two kids are seen playing cooperatively using one gamepad, while a third kid stands by listening to the resulting soundscape through headphones.

Shiny, by Ateo

Shiny, presented by Janina Woods

The final presentation was by Janina Woods who is co-developing the virtual reality game, Shiny, at Ateo with Sebastian Tobler. Gameplay-wise, Shiny and MODSORK seem very similar although the music drives gameplay in Shiny, whereas the opposite happens in MODSORK. Players of Shiny must eliminate glowing cubes that are dynamically generated by music. Cubes are targeted using a reticule that is controlled by the player’s gaze using the virtual reality headset.

Woods discussed design challenges specific to virtually reality in the form of attention bottleneck. Woods has found that players are so overwhelmed with information, due to the increased level of immersion in virtual reality, that they are unable to notice anything other than the targeting reticule—even if the information is glaringly obvious and positioned directly beside the reticule. This has led the development team to integrate UI information into gameplay objects using color and shape. 

Woods also talked about the relationship between interaction and music, in response to a question posed by Masaya Matsuura: how is scoring and music expression affected with regard to blocks cleared and short and long ranges? Woods explained that blocks destroyed at long range give you a heightened sense of accomplishment but don't influence the score in any way. Each group of blocks has their own sound effect in 3D space, so destroying blocks at varying distances creates a different 3D soundscape. The effect adds to the sense of immersion and audio consistency.

Summary

The games presented during the IGDA Demo Night explored the boundaries between music, gameplay and interaction. We learned that PaRappa the Rapper, furusoma and Shiny fall into the category of games in which the music drives gameplay. MODSORK takes an alternative approach by having the gameplay generate music. While Panoramical, and Don’t Go – The Purest Of All Sounds treat music like an instrument to be explored through interaction of varying degrees.

Interestingly, the presentations highlighted a second question concerning authorship: who is the creator of the game’s music and sound effects? Panoramical, and MODSORK provide players with the most freedom to be co-authors alongside the game’s creators. While games like PaRappa the Rapper, furusoma, and Shiny give players a limited amount of freedom to create sounds outside of gameplay but the consequence, of course, is a negative impact on scoring. Don’t Go – The Purest Of All Sounds—having no gameplay, per se—firmly placed music authorship in the hands of the composers.

The evening concluded with the audience demoing games that were featured in the presentations as well as several others by Switzerland-based developers whose games deserve an honorable mention: Splash Blast Panic by KwaKwa (which has been accepted for Steam Greenlight), Drop the Bodies on the Pool by Jonay Rosales González, and oOku by ooku.org.

A sincere thank-you to the presenters and event sponsors, Pro Helvetia, and SAE Institute Zurich; to the kind supporters that helped out in various ways: Ebrietas, Ai Kinugawa at NanaOn-Sha, Philomena Schwab (Niche—a genetics survival game), Beat Suter and René Bauer at ZHdK Gamelab, Kenji Ono at IGDA Japan, Sanae Hiraya and Yuko Sawa at B-Sprout Japanese translation services, Robbert van Rooden, Sabine Schweizer, and Oh! sweetest thing café.

For more information about the IGDA Switzerland Chapter please visit our info page on the website of the Swiss Game Developers Association (SGDA), with whom we are partnered.

///

About the author—CHRIS SOLARSKI (Solarski Studio) is the IGDA Switzerland Chapter coordinator, an artist game designer and author of ‘Drawing Basics and Video Game Art: Classic to Cutting-Edge Art Techniques for Winning Game Design’ (Watson-Guptill 2012), which has been translated into Japanese and Korean. He has lectured on emotions and storytelling in video games at international events including the Smithsonian Museum's landmark The Art of Video Games exhibition, SXSW, Game Developers Conference, and FMX. Chris’ projects at Solarski Studio continue to explore the intersections between video games, classical art and film to develop new forms of player interaction and emotionally-rich experiences in games. His second book—‘Interactive Stories and Video Game Art: A Storytelling Framework for Game Design’—will be published by CRC Press late-2016