Opinion: Middleware For Music

[Xiotex Studios' Byron Atkinson-Jones, explains why having no talent in music production might not be as big a barrier as it seems when creating your game's soundtrack, in this #altdevblogaday-reprinted opinion piece.] There is one aspect of game development that I historically had problems with and that was audio. I would concentrate all my efforts into getting the engine, game code, and graphics right, but audio would take a back-seat. In fact, in the earlier years of my indie company, I released games without any audio. It became rapidly apparent that I was leaving out a vital component. When it comes to game development, I was obsessed about being able to do absolutely everything myself. Not because I don’t trust anybody else to do it, but rather because I have an autotelic personality, and that means I have an insatiable thirst to learn everything. So, when it came to doing the music and sound effects for my games I just had to learn how to do it. It’s a long arduous path, but I’m now at a stage where I think I can do some basic composition. However, if you were to ask my wife, she would say that I haven’t quite gotten there yet. As with all other areas on game development like coding, any musical ability I now have is self-taught, and my path of musical education took me along lots of different routes, some involving hardware like dynthesisers and others through software such as trackers, Reason, and Ableton Live. My education is by no means complete, and I suspect it will be a long on-going part of my life as I learn more and more. Like anything else in life, it requires practice, practice and more practice to remain proficient. Hardware Or Software? In my earlier games, I played live on a keyboard and recorded the audio straight through an audio interface into a sampler like Audacity. I then converted the sample into an MP3 file that I played in games. There was no track and my timing isn’t perfect. The pieces are all improvised, which really shows as some of them go wrong at some point. I’m getting better at playing the keyboard, but it will be a number of years until I am proficient enough to be able to play it accurately; my fingering technique is all over the place. In some of the pieces, I have more than one piece of hardware synchronized to either provide the beat or the bass lines. As time went on, audio production software, or DAW’s (Digital Audio Workstation), got better, and now I tend to use them rather than play live. This has changed how I work because now the piece becomes more considered, rather than me letting chord progression and intuition guide me. I still just sit at the keyboard and jam every now and then just to keep the practice in. My particular DAW of choice is Ableton Live. I also have Logic Studio 9 and Reason 5 – both equally capable pieces of software, but I find Live has the edge in ease of use and ability to simply experiment in composition. If you are starting out, the good news is that a DAW doesn’t have to cost the earth, and there are some very capable cheap versions out there. In fact, Ableton makes a cheaper verion of Live called Live Intro. It doesn’t have all the features of the full version, of course, but in order to make the kind of music I did for this article, you don’t need the full version. Use the sample libraries Early on in my musical exploration, I came across the website of a company that made sample libraries, and they claimed their products were used on TV shows like CSI. As a CSI fan, this drew me in like a moth to a light, and I shelled out the cash to get it. I wasn’t disappointed, and since then I went on to get most of the products they put out since they are of such high quality. The company is called Sample Logic. They are called sample libraries, and you might be mistaken for thinking that means it’s nothing more than a large collection of .wav files, but in reality they are so much more. They make use of the Native Instruments Kontakt player, which allows library creators to more than simply play back samples; they can layer, add effects, and generally morph sounds over time.  Sample Logic has done some amazing things with this player to produce its vast libraries. They are not the only company doing this, in fact two other companies called Heavyocity and Spectrasonics make some amazing sound libraries that I also use. As an example, in one of my games, the music was created using Spectrasonics Omnisphere on its own. I wanted to demonstrate the power of these libraries but to do it in the manner that does not require a huge amount of musical experience, so I went through looking for five instrument sequences I could string together to make a very basic track and not require any further mastering or effects. Don’t You Still Have To Be Proficient In Music Theory? So, how are the various sample libraries helping me, don’t I still have to know a lot about music theory? Music theory will help me out, and I still continue to read everything I can find about it, but in the meantime these libraries are so flexible that all you need is an ear and some ability to tell when something doesn’t sound right. Even though these libraries can seem to pull some musical miracles, it’s not going to be a substitute forever for a lack of musical knowledge/ For instance, what if you wanted to write a piece that doesn’t quite fit into the spectrum of genre that the libraries outwardly appear to support? For that, you might need a bit more help from music theory. I have no doubt that while something I put together from these libraries might be passable, in the hands of somebody who is proficient in music theory and practice, amazing things could be done. Until I reach that stage, I am happy to keep experimenting and refining. Decomposition The piece I put together for this article is an example of how simple it can get. The whole piece of music is made up of five MIDI clips in Ableton live. A MIDI clip is simply a block of notes to be played in order, so these five blocks form up the whole block of music. The structure of the music is simply the order in which these clips are played and turned off. Ableton Live allows these MIDI clips to be arranged into scenes, and inside these scenes you can turn clips on and off. So, if you listen to the piece of music, blog you will notice towards the middle end I turned off the guitars to let the lead loop play in relative isolation, and then I turned the guitars back on again. I have already said the contents of these clips are the notes that get played to make up the music, but given how complex the example music is, just how simple can that get? Stringing notes together to make music can be complicated, requiring lots of knowledge of rhythm, scales and harmony among others. Well, in my piece, each clip contains exactly one note, and given that there are five clips, that means the piece of music is composed of five notes in total, no more, no less. Yet when you listen to it, there are clearly more, a lot more, what gives? This is the power of the sample libraries and the Kontakt player. Each note triggers a sequence to be played in the Kontakt player, and that sequence can be very simple or as complex as Sample Logic can make it. To show you how this piece was put together, I made a screen-cast. In the first half, I go through each of the clips, showing the notes contained within them (which, as I already pointed out, consists of just one), and then solo that particular clip so that when it’s played you just hear that one clip. Once each clip has been played individually I quickly go through the entire piece by playing the scenes that make it up, which gives the music its structure. You can find it here. This piece was never designed to win awards or to be taken seriously as a piece of music but to show how simple it is these days to make use of some powerful sound libraries to make what passes as music. To be truthful, it could have done with some proper mastering to balance the stereo mix and levels out, but since I wanted to show how simple this process could be, I decided to leave it as it is. That’s Got To Be Cheating! Is it cheating? I struggled with that initially, so let's examine it a bit more. If I were to go to Turbo-squid and download a 3D mesh to use in my game, is that cheating? What about if I purchase a texture set to be used? Sound effects? Each item on its own is just one small element in a larger piece that makes up your game; it’s the combination that makes the piece, not just the components that make it up. Entire genres of modern music were born from sampling other people’s records and putting them together into new arrangements. In fact, many artists out there use stock samples that come with DAW’s in their songs. The artist Rhianna used a loop that comes with Logic Studio in her song Umbrella, and once you hear the loop on its own, you can’t but help notice it. One UK periodical on modern music production went so far as to claim that the modern use of samples was not too far removed from using a session musician, i.e. somebody who came into the studio to record parts that you don’t have the in-house capability to play. I’m not so sure I fully subscribe to that, but the principle is sound enough. There’s also another set of factors that comes into play and that is of time, money, and ability. If you have a game that you have to make but you don’t have the time or ability to construct an engine to use, what are you to do? Do you spend time and money to obtain the knowledge, or do you go out, research the available engines out there, and buy the rights to use one in your game? This is partly why I call the types of sound libraries I have been talking about middleware for music. They take a large part of the necessary knowledge and time out of the process of learning how to produce that kind of music, and allow me to get my games out in a reasonable time frame. Ultimately, my goal is to make music that compliments my game, and I will use whatever gets me to that goal, and to be honest these tools exist to be used to make music, so why not use them? In the end, it’s up to you to make a decision about which path you choose. Why Not Just Pay A Musician To Make The Music For Me? Good question, there are a lot of very talented people out there that are more than willing to make music for my games for a price, and in reality, that price might be a lot cheaper than paying out for the software and hardware. I would certainly recommend going down this route for the ultimate in timely music production, depending of course on the artist you find to do it for you. It would certainly take out some of the hassles of worrying if it was good enough. However, having said that, I personally find it difficult to describe what kind of music I am after. With art, you can do a quick sketch to get your idea across, but with music, the most you are going to have is references to other pieces of music. This is where the really good musician comes into play though – they should be able to take your game, play it,  get the feel of the kind of mood you are trying to capture, and then create some music as appropriate. When I sit down with my keyboards and sample libraries, music is allowed to evolve and I am able to shape it into something that I had visualized. Sometimes I can get a bit carried away and the music I create has nothing to do with the game, but it’s fun. As I said above, I am a bit autotelic, so I have this burning desire to learn everything I can about all aspects of game development, not just the code, but music and artI I have also enrolled on art courses to improve in that aspect, too. It would be great to one day release a successful game and be able to say I did all aspects of it. So What Was The Point Of All This? I guess the message I was trying to get across is that having no talent in music production should not be as big a barrier as it seems. Software and even hardware exists that makes the whole process of creating music a lot more open to the masses without the need of years and years of practice on an instrument. It’s not going to replace such dedicated work, but it does allow you a foot in the door, which might otherwise have been unattainable. The technology exists to get those of us who would struggle to create music a step closer to being able to produce a mood piece for our games without too much blood, sweat, and tears. It’s not a substitution for a really good musician, but if it’s just you doing it, then it’s there to be used. [This piece was reprinted from #AltDevBlogADay, a shared blog initiative started by @mike_acton devoted to giving game developers of all disciplines a place to motivate each other to write regularly about their personal game development passions.]