Composing Adaptive Music for Video Games Using Elias

“But I'm just a soul whose intentions are good

Oh Lord, please don't let me be misunderstood”

-  Bennie Benjamin, Gloria Caldwell, and Sol Marcus

“To be great is to be misunderstood”

- Ralph Waldo Emerson

 

Introduction

Imagine you are a proud adventurer walking through a forest. Tranquil forest-y music is playing.  Then you spot a cave, and you decide to enter.  The music changes to a tense tremolo string theme.  Then, all of a sudden, out of the darkness jumps the biggest monster you have ever encountered and battle ensues.  The music grows gradually more fierce and frantic until you are within an inch of your life.  Being a proud adventurer, however, you succeed in slaying the beast and the music changes to a victory theme.

How is this kind of music written, and how is it implemented in video games today? Welcome to this first in a series of articles about composing adaptive music for video games.  In this first article, I am going to talk about the two main approaches to writing adaptive music, horizontal and vertical.  I discovered a really interesting new tool called Elias that will help to illustrate my points and allow me to show you some concrete examples.  

To demonstrate Elias, I composed an adaptive score which I call “Elias Space Theme.” Here is an MP3 version of the theme, which was recorded directly from Elias:

Introducing Elias

Sometimes, something so innovative comes along that it is misunderstood.  Such is the case with Elias Software’s Elias Engine and Elias Studio. I first started using Elias Studio last year when it was just announced and in version 1.0.  I immediately fell in love with its intuitive interface and powerful features. I was able to turn a linear composition into an adaptive score ready for a video game in less than an hour. Other middleware products that I have used have taken me weeks to learn how to do something like Elias can do right out of the box. Elias is not designed to be a replacement for middleware such as FMOD or WWISE, but rather as a complement to them.  It has two main features, Elias Engine and Elias Studio.  The best part - Elias Studio is free for composers to download and use and we can start creating adaptive scores that sound rich, complex, and natural, as if we had a conductor and a full orchestra in front of us. Elias Engine can be licensed by game developers and has a very friendly fee structure for indie developers that scales up for larger projects. There is a new Unity plugin, and it can be used on both Mac and Windows.  It is the first audio middleware product that I am aware of that was created for composers by composers.

Let’s pause for a moment and take a look at Elias in this overview video:

Elias is a tool that composers and game developers can use to manage complex scores that play the appropriate music at the appropriate time based on the events and actions in the game.  It is a win-win for composers who put so much thought and creativity into their music; for game developers who want to create great games and for gamers who would get an experience just as immersive as a movie score.

Elias is designed to handle live music rather than programmed or procedural  music. It is able to adjust seamlessly and adapt between parts of the musical score in a way that is imperceptible to the listener.  Elias behaves like a conductor, and the game tells Elias when to move smoothly between parts of the score. Elias manages when and how the parts change. Some parts change immediately, and other parts change at musically appropriate bars and beats.

In my estimation, having worked with Elias for many months, it fully achieves this goal and surpasses it beautifully.  It is lightweight; cross-platform; stable and easy to use and implement for both composers and game developers.

Misunderstood

What about that misunderstanding?  Well, when most people look at Elias they tend to think it is a loop engine, similar to Ableton Live.  Indeed, it has a similar, familiar layout but that is where the similarity ends. There are two main ways of thinking about adaptive music composition for video games; vertical and horizontal.  In horizontal composition, the music is often written in short 1-4 bar chunks that can be elegantly switched around and interchanged. For example, if two bar chunks are used, then for every two bars a new fully mixed set of two bar music would be played.  A more common approach is the vertical layering, which has two main flavors.  Vertical layers are analogous to stem mixing in standard DAW’s with the main difference being that not all tracks are meant to be played simultaneously.  The two flavors are additive and interchange.  In the additive approach, layers are added one on top of another, and in the interchange approach, layers are swapped out.  Elias employs the latter approach, and this turns out to be one of the most flexible and versatile ways to write game music.

However, Elias goes one step further and allows you to set the way in which these layers are switched and the timing for each track.  For example, if you have chords changing every 2 bars and a melody line that is 8 bars long, then you can switch chords every 2 bars, and your melody line every 8.  The secret sauce is in how these layers switch - which can be exact (or “strict” in Elias terms) or fluid, which is determined heuristically by the Elias engine.  I will explain this further in another article.

In addition, Elias currently has two modes; objective and exploration.  In objective mode, the intensity typically builds from low to high, for example in a game situation where a battle gets gradually more chaotic and intense.  In exploration mode, layers are played randomly and interchanged with one another to allow for tremendous amount of variation with a relatively small number of “tracks.”

Interface

The interface is clean and elegant with a lot of features buried beneath the surface.  Once you open Elias Studio, you can easily add tracks and then simply drag audio files into the interface and arrange them.  

There is a global settings dialog which lets you modify the overall settings for the project, such as reverb settings; whether you want an “Objective” or “Exploration” theme, and so on.

There is also a settings dialog containing modifications such as fades and “agility” which tells Elias how and when to switch between layers.  There is also a reverb setting that can be different for each track if desired.  

Also pictured here to the right is the “stinger” tracks which lets you set up any number of stinger types and variations for your song.

There are a full set of tutorial videos on their website, and I encourage you to watch those as they go into great detail about all the wonderful features of Elias Studio.  There is also a special type of track called “Stingers” which lets you add stingers such as cymbal crashes; orchestra hits, or drum rolls at any point in the composition, such as on level changes.

Elias in Action

Let’s now take a deeper look at Elias in the next video:

The vision for this composition came from watching the TV program Cosmos, and from playing video games such as Master of Orion, Elite Dangerous and MassEffect.  I wanted the theme to have an idée fixe, or theme that would be played by various instruments and synths as the theme evolved over time, along with variations for the different scenes. I wanted it to be ethereal, infinite, and accompany the beauty of flying through space and entering various solar systems.  This was not designed to be battle music, or haunting in any way, so I decided on a major chord harmony that evolved through the circle of 5ths.  This allowed the music to move through six different keys and return to the beginning, analogous to a Shepard Scale which is used frequently in video games.  The Shepard Scale is a kind of musical illusion that gives the infinite staircase feeling you might be familiar with from Super Mario 64.

In terms of tonality, or timbre, I wanted the composition to be alive with full orchestra, but also to have an ambient evolving synth bass.  So, imagine if you will a full orchestra jamming with synths and you'll get the picture.  

To learn more about the composition, let’s take a look at the Pro Tools Session that I used to mix the song:

Conclusion

This article is an overview of the two main types ways composers write adaptive music for video games, horizontal and vertical.  In the videos I go into detail about the most advanced type of writing adaptive music, the vertical interchange method.  One tool designed to help composers write and implement this type of music is Elias.

Elias is an elegant and powerful tool for creating complex adaptive scores for video games.  It could even be used for scoring videos and films as well. Other middleware solutions are very complicated to learn and even more complicated to master.  None of them currently offer an easy solution to do the types of things that Elias can do out of the box.  Elias is lightweight, non-destructive and can switch between musical ideas on the fly and make musical transitions seem effortless.  Elias is designed to handle live, non-quantized music and play it back with incredible variety, smoothly.  

It is free for composers to use and has a very flexible licensing plan for developers.  There are opportunities for composers to create themes and then sell them through the Elias website, and there is also a collection of amazing themes available now on the website.

Elias is now in version 1.5, and is constantly undergoing feature upgrades and has evolved in many exciting ways in the time that I have been using it. The developers have recently received significant funding, and are working around the clock to add better functionality and new  features, such as the recent Unity integration.  They are very approachable, friendly, and willing to listen to new ideas and consider adding new features all the time.  Best of all, they are here to stay and have created an amazing tool that I have really enjoyed learning and using on my projects.

In the next article I will go into writing adaptive music for a virtual reality game called “Stampede” which is written in Unity for Oculus and other upcoming VR headsets.