Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs or learn how to Submit Your Own Blog Post
In honour of Advanced Input week I look at controlling a game using sound alone.
As a part time game developer, full time composer, I'm always interested in blending the worlds of music and games. I recently tried to create a video-game that was controlled solely by the sounds a player made.
I settled on using the parameter of pitch. You can see a video of my little proof of concept below:
And if you wish download (from dropbox) and play it here!
Although it is not very often utilised, the potential for using sound to control a game (outside of simple pedagogy) is really exciting and actually not very difficult to implement. An instrument or voice offers some very interesting control possibilities that a game pad or even a computer keyboard can't deal with.
For example; the sheer number of easily accessible inputs - pretty much anyone who knows how to vaguely play an instrument will have at least a grasp of where the notes of their instrument lie, and this means they have access to a controller with potentially hundreds of inputs that are much easier to use than on a standard keyboard. Imagine an RTS or RPG with its myriad of controls and quick binds (awkward and unintuitive) - translate this complexity onto a violin or piano and all those binds become accessible and more importantly, fluid. This frees these games to change their pace dramatically, transforming their complexity into an action oriented realm, maintaining their emergence and depth while being able to process user input at a break neck speed and in a fluid manner that mimics the moment to moment viscera of the action genre.
Other possibilities:
Timbral Effects that mimic game action: (1) In a platformer you could use upwards or downwards slides mapped to the players' jumping curve (2) Using clicks or pops in a shooter mapped to fire.
Dynamic effects: Using relatively volume. Imagine playing a game like "Flower" but where making a wind sound with your voice at relative volumes is mapped to the movement.
Chordal analysis: Multiplies the possibilities of single pitch analysis although these inputs are slightly more difficulty to use.
Other Musical Possibilities: Trills, turns, pre-mapped musical fragments that could be inserted intertextually into the game to create a really rich tapestry from which to control the action.
However there are a few warnings to bear in mind when trying to create this sort of game:
Your player is constantly making a sound and therefore you either have to completely forego music in your game or work very hard to synchronise your game audio to the sound your player is making.
With pitch, dynamic and timbral analysis there is often a slightly longer input delay than with a controller - this can become negligible with very good coding - but it is something to be aware of.
Sometimes playing music can lack the visceral effect of using a controller, especially when using the voice.
The difficulty of games controlled by sound has to be tuned down slightly as the general skill level of players using these systems is relatively low compared to a game-pad or keyboard.
With all of this my main aim was to challenege people simply to think about the possibilities of controlling their games through a medium that can enrich their aural perception or musical sensibilties rather than just the coordination of their fingers.
One of the things I haven't really got into is how these different control systems really impact design - I've mainly looked at mapping existing control schemes onto sound input. However, if developers really start using this technology then the whole ludology of sound games will have to shift. I have no idea what this could lead to, but I can imagine games which fall more and more into the fantastic flow of music and pure sound. Where the aesthetics of the play experience stem from the expression of creating beautiful noise and are encapsulated as much in the game's systems of input as in the rules.
If anything, games which use true musical and sound input need to escape from the tyranny of the rhythm game, of the (infamous) Guitar tutor and other pre-defined genres and carve out their own world of expression. If you are a developer try it out! I'd be excited to see the results. I made a post about the specifics of doing this in unity in a similar post on my other blog to help get people started. It is amazing how something as simple as the method of interaction with a game can completely change the meaning of a game experience. It is something that all game developers should play with and experience.
Declan Kolakowski : @infiniteoctagon
Read more about:
Featured BlogsYou May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
