Understanding Synaesthesia for Better Games

As a creative director, it is often a challenge to properly emphasize the appropriateness behind creative choices, and indeed why certain creative choices are better than others.  As is common in communicating, I find myself using terminology out of its normal context and use it in a similar context in order to effectively describe it. 

Drama for instance is a kind of performance (as in acting) and yet, it is quite common to describe images or music as “dramatic”.  We readily make judgement across media in real life.   In the game world however, where the visual can be so limitlessly altered, finding a harmony amongst all elements can be challenging.

Synaesthesia is a disorder of the brain which causes sensory inputs to be perceived by multiple perceptual mechanisms at the same time.  What is most commonly cited as a symptom of synaesthesia is the ability to see numbers as colors.  So, for instance the number five might be seen as the color red. 

As science studies this disorder, is seems to me that in some way the brain is exaggerating what is a normal function.  Synaesthesia is said to be more common in artists, musicians, poets, etc.  As a creative person (a classically trained artist) I have found that I do in fact perceive the world differently from those who are less artistically inclined. 

I should be clear though that I do not have synaesthesia, but I do have this innate ability to make analogies and thereby understand things a bit easier than most.  This perceptual difference allows me to make obvious connections on creative choices that others have difficulty grasping.  That said, I believe that part of it is mental training rather than strictly genetics.

In university, there were exercises in studio to help us “free our minds”.  We’d be asked to paint what it is to be an N for instance or we’d be asked to paint emotions.  In fact, this is something that is part of the cultural vernacular of the west.  Terms such as “red with rage” or “green with envy” are testament to the notion that humans naturally make cognitive connections between seemingly unrelated items, in this case colors and emotions.

This brings us to the issue as it relates to game development.  As with everything that is human, our capacity in this area is on a sliding scale.  Someone with Synaesthesia represents an aberration of the function.  Someone like myself, or a musician, or a poet represents a person on the more developed or capable end of the spectrum.  This means that there are also people on the other end of the spectrum to whom these connections are not so easily apparent.

I ran into this situation a few years ago and I found it to be terribly difficult at the time.  I was in a discussion with a co-worker from another project about my opinions on that project.  Specifically, it was a discussion about the story accompanying a game.  My feeling was that the story was inappropriate and would decrease the perceived quality of the game. 

To me, a story is quite vivid.  I see quite clearly what is being communicated to me in text.  At the same time, visuals communicate to me.  I’m sure you’ve heard people say “that image really speaks to me”.  In this case, the story communicated a visual style, not in so much in the actual words, but in its art of writing.   

Likewise, the art direction communicated a literary style.  These two lines of communication were in conflict rather than in harmony, and although they might both be equally amazing in execution on their own, the fact that they were not in harmony would detract from the experience once they were combined.  It was here that I realized that the obvious clash that I perceived was at the same time completely nonexistent to my co-worker

I searched for a term because I’m sure this isn’t a new concept but I did not readily find one, so let’s use the term media harmony.  When the “art” of a particular medium is in harmony with the “art” of another medium, the result is a media harmony and when the arts of two mediums are in conflict, you have media disharmony. 

This media disharmony affects the player even though he may not be conscious of it or unable to verbalise it.  The player will have a less enjoyable experience than he would if there were a media harmony.  Media harmony will lead to a higher subjective judgement of the game while media disharmony results in a lower subjective judgement.

Looking at movies for a moment, successful movies hinge upon casting the proper actor for a role.  It may seem subjective, but I believe it is more about media harmony.   The correct choice of actor will properly convey the character portrayed in the script.  In other words, the visual media and the literary media are in harmony.  Often, a script is written with a particular actor in mind and I would argue it is for this reason.

You’ve been in a room when two people are arguing (if you’re like me it was most likely your parents).  Game media has a conversation with the player.  It communicates to him.  The player hears it, or sees it, or reads it.  When two media are communicating conflicting information, it’s tantamount to an argument with the player in the middle.  It’s not a comfortable place to be.

The best way to achieve media harmony is to envision what an element would be like as another medium.  As with my university exercises, take your game’s story and try to picture it.  What does it look like?  Is it bright or is it dark?  Is the world realistic or surrealistic?  Look at your art and try to listen to it.  What does it sound like?  Caption the story of it.  What does it read like?  Is it a drama or a comedy?  Try to put the other elements of the game out of your mind during the excercise.

In time, your brain will awaken and make connections more readily.  You can even use this as an exercise within your team to gauge the media harmony of an element.  You can then document the responses from various team members and present this feedback.  It’s important to not ask “does this go with that?” 

You must have responses that deal strictly with the media in question and compare the responses from the different media evaluations.  This is important because when something is in disharmony, it may not necessarily be “bad” content.  It might actually be quite good.  However, trying to convince your content creator that the great work he or she has done is invalid will be challenging. 

Whether two elements go together or not is generally thought of as completely subjective.  If you can show that the story, for instance, evokes happy-happy imagery and then show how the game art evokes solemn stories, you can then work towards a harmony.

Take a closer look at the games that are generally considered the best in their genre and dissect them with media harmony in mind.  I think you will find that in most cases, there is a synergy in all elements.  The creators may have arrived at this through innate talent or dumb luck.  In any event that doesn't mean you can't learn a few tricks to help arrive at a positive outcome for your game.