Interactive Audio in Black Knight 2000: The Importance of Integration

I recently had a blast from the past, when Kotaku published a video entitled “The Five Best Game Songs Most People Haven’t Heard” and decided to put the music for a game I composed some time ago on the list.  Having been recently re-released as an iPad simulation in Pinball Arcade, the music for the pinball machine Black Knight 2000 (BK2K) struck the reviewer as being worthy of dredging up from obscurity.

This made me ponder the reasons why this particular game made the list.  What was it about this game that makes people remember the sound & music.  And I’ve come to the conclusion that, although the music itself might be kind of catchy, it’s really the full package, implementation and tight overall synchronization between the music, visuals and the gameplay itself that make it memorable.

As I watched the video, it reminded me of how interactivity in game music and sound has changed over the two and a half decades since that game was released, not necessarily always for the better.  Certainly production values have  increased exponentially and no one would even think about going back to the FM synthesis chip technology and limited memory days.  (If I recall correctly, the entire audio memory budget for that game was around 64kilo bytes.  Kilo as in ‘thousand’).  Still, one of the benefits of the primitive synthesis-chip tech was tremendous flexibility in how we could sculpt the entire music and sound package to the game.

Interactive Music & SFX, circa 1988

Believe it or not, pinball machines require a lot of music.  It’s not uncommon for a game to have 25 or more minutes of unique game score.  That’s because, due to the complexity of pinball rules, pinball machines can have a lot of different gameplay states.  The main game states are the progression from normal play, “multi-ball ready” (a tense ‘one shot away’ mood), Multiball (frenetic music for frenetic gameplay with multiple pinballs flying about), and then “jackpot scored”, a celebratory piece.  In addition to these, pinball machines include numerous mini-games.  These are theme-based challenges, which require the player to make certain shots or combinations of shots, often within a limited period of time.  “Beating the game” in pinball  means not just getting a high score, but going through all the main game stages, and reaching and beating all of the minigames on a single quarter.  Each of these mini-games requires its own music, and it’s not uncommon for a game to have a dozen or more such mini-games.

A big challenge from a musical scoring perspective is that the game can switch between these states in an instant, based on a single target hit.  This means the music has to be able to easily and smoothly flow, yet turn on a dime; one moment you are in multiball-ready, the next you have started a frenetic ‘shoot as many ramps as you can in 30 seconds’ minigame.

A design precept for BK2K’s music and SFX was “once you press START, the music should never stop and never miss a beat.”  Each and every song to song transition is on a half measure or measure boundary.  That lasts until after the game is over and the end screen is displayed.  We had a song-matrix system which would allow you to select various transition segments depending on which song was playing, where in that song you were, and what your destination music was, always on the beat.

Blurring the line between music and SFX and deep audio integration

One particularly fun feature of the music and sound system was the ability of the background music to change the character of sound effects as they were layered on top.  Our music system allowed us to tag the score with chord and key markers, giving the entire audio system global knowledge of the current underlying chord and tempo.  Some of the sound effects would use this information to change what pitches they played, so as to fit right into the underlying music.  In fact, if a long sound effect started playing, and then the underlying background music chord changed mid-effect, you could hear the SFX instantly re-transpose itself mid-playback to match the proper underlying chord.  (you can hear this especially well on the sweeping “ball lock” sound effect in BK2K).

In addition, we could tag certain sound effects so that they would queue up to synchronize on a musical boundary, most typically a 16^th note.  The pop bumpers do this, for example.  We made the pops a tom-tom-like sound, so when the ball is being battered around, it sounds like an impromptu drum fill.  The sound effects also know the underlying tempo, which would allow the creation of sound effects that might consist of a 16^th note run, and have it actually start on a beat or beat subdivision, blurring the line between “background score” and “foreground sound effect.”

Integration, Implementation and the Power of Audio-Visual-Gameplay Synchronization

Perhaps BK2K’s biggest audio feature, though, isn’t really ‘audio’ at all — it is the very tight synchronization and coordination with the rest of the game from visuals to gameplay flow itself.  We were extremely fortunate for a couple things going for us musically on this game.  First the game designer, Steve Ritchie, is also a musician.  He’s a guitar player and co-wrote the music.  So he “gets” what music and sound can add to the game. 

Secondly, the programmer also completely “got” sound, and put the time and effort into the game to integrate it properly and tightly.  The programmer, Ed Boon, (who went on to design and create the game, Mortal Kombat, after finishing BK2K) was a stickler for detail in all aspects of the game, including audio.  He certainly put in plenty of overtime ensuring that audio, visual and gameplay were all on the same page and tightly synchronized and integrated into a complete package. This support from both design and programming allowed for a very tight integration between sound, visuals and gameplay.

Pinball machines are very audio/visual games.  The playfields are filled with bright, quick-flashing lights, as well as displays capable of displaying animations and images.  Virtually every sound in the game has a tightly corresponding visual effect, whether it’s in the score display or is a light or target flashing on the playfield itself.  The music system let us control the lights on the playfield directly from the musical score!  We would tag our music with specific markers which would fire off the lights, in tempo and with precise synchronization with the accents in the music.  If you look very closely, you will see many of the playfield lamps and flashing lights are flashing to the beat or on specific musical accents.  Imagine that—the game relies on the composer to trigger and control the visual effects!

Music…Wait for it….

The desire to have the gameplay/aural/visual experience be holistic was so great that there are points in the game where the gameplay actually momentarily pauses, and waits for a ‘musically appropriate’ time to continue.  For example, in pinball, after the ball drains, you generally look to see your score and any ‘bonus points’ you earned during that ball.  In Bk2K, after the ball drains, the game waits for the next 2-beat boundary before it puts up the “BONUS” display.  And it puts it up exactly on the beat in sync with the music.  Similarly, when the game needs to kick the ball onto the plunger for the next ball, it likewise waits for the next 2-beat boundary, and kicks out the ball in precise time with background music.  Despite being a very fast-paced table, BK2K is filled with places where the game waits ever so little for a musical boundary to continue the gameplay.

The result is a completely synchronization and integration of gameplay, visual effects and aural experience.

Back to the Future

One of the highest rated talks from last year’s GDC Audio Track was Guy Whitmore’s talk on interactive music in Peggle 2, where they clearly are pushing the boundaries of current-generation interactive music, often blurring the lines between sfx and music.  But somewhere in the back of my head, I think back to the days of the old BK2K music and sound system and wonder when we’ll get back to the tech we had two and a half decades ago.  [note: Guy Whitmore discussed tightly integrated sound effects and interactive music at GameSoundCon in October in LA, in a talk entitled, “Interactive Music in Modern Games: Having it All!”]

Coda

So although I’m quite proud of the fact that kotaku called out the Black Knight 2000 music as being noteworthy, and that various pinball music aficionados have sent me youtube videos of themselves playing the BK2K bass line or covers of the themes over the years, I have a hunch the reason they find the music so compelling is its role in the complete game/audio/visual package and the attention to detail in integration.  And as we learned back in 1988 with Black Knight 2000, that’s something that has as much to do with support from the game designer and the rest of the game team as it does with the music itself. 

Addendum on Tech & Harmony

The music and sound system for WMS games (the company that made Black Knight 2000) was primarily written by Bill Parod and Chris Granner, and was later modified by myself, Dan Forden (who also provided music to Bk2k) and others, and was written in 6809 assembly language.   The sound system comprised a 2-MHz 6809 processor with 8k RAM controlling a Yamaha YM2151 audio chip, which could play eight 4-operator FM voices.  We could also play back one channel of 8-bit PCM (sampled around 7 or 8kHz if I recall), which was used for a monophonic drum track.  The singing was actually generated from the speech system, which was a truly horrid technology called “CVSD”, which was an analog compression chip (HC55536), using 1-bit samples at around 8kbits/s.  In fact, the ability of the music system (one computer board) to control the speech chip (which was on a wholly separate processor board) was thought to be impossible until Ed Boon finagled a way to make it happen; without that, there would be no ‘choir of angels’ or taunting knight in the BK2K themes. 

Memory was extremely limited, which is why if you listen closely, you can hear the same choir “aaah’s” being re-used in different parts of the soundtrack which occur deeper into the game.  Listen for an E-minor chord sung by the choir being used 3-ways—as an Eminor, as a Cmaj 7 and also as a “I 6/4” chord, acting kind of like a Bsus.  The same Em chord also appears in the “multiball ready” music.  Careful listeners will hear a re-use of the phrase “you got the…” before both the words “power” and “might.”  Yes, there are definitely some things would never want to go back to!

Brian Schmidt is a 27-year game audio veteran and an independent Game Audio Consultant, Composer and Sound Designer at Brian Schmidt Studios, LLC and is founder of the audio game company,Eargames.  He is also the founder and Executive Director of GameSoundCon.  Brian sits on the GDC Advisory Board and is President of the Game Audio Network Guild.  Brian just finished sound design for his first pinball machine since 1998, Stern's The Walking Dead.