The Shifting Saga Of Sound Shapes

[It took four years for Everyday Shooter developer Jon Mak to get his next game, Sound Shapes up and running -- but it's shaping up to be one of the PlayStation Vita's most anticipated titles, and he discusses it inside.]

Everyday Shooter was an early star of the PlayStation Network lineup -- a game that promised that Sony was as serious as Microsoft at making sure its service was home to some of the most appealing and creative indie games available.

But developer Jon Mak went silent after that. It took four years, and teaming up with musician Shaw-Han Liem to create Sound Shapes, which is making its way to the PlayStation Vita and has, in fact, become one of the fledgling handheld system's most anticipated titles.

In this interview, Mak and Liem talk specifically about the challenges that lead to the project's slow development, and discuss the technology that allows for interactive music to form the key of the game's platforming experience.

Let's talk about why it took so long to get to your next project to the point where you are comfortable with releasing it.

Jon Mak: Well, after Everyday Shooter, I met Shaw-Han and he still had a job. So I had to convince him to quit the job, and then go on this four-year-plus hell ride with me, making a video game.

We actually started off working on just, like, music visualizers for shows in Processing, and it sort of grew from that to be like, "Let's make a really small video game, sort of in the vein of Everyday Shooter." Except it wasn't shooting; it was like a beat 'em up or whatever, and then we just kept one-upping it.

It was just thinking about how the player could participate in making music while playing a game, and about two years ago, we came up with a prototype where it was like, it's a platformer, and you make levels, and that became Sound Shapes.

The hard part is, we don't want to make any concessions on the gameplay side or the music side. So it's really hard. When everything you put in the game has to have a gameplay function and a musical function, it's really hard to do that, because you can make something that sounds really cool, but then the gameplay might get fucked up. Or you might make something that plays really well, but how are you going to make that musical?

We had this thing with moving platforms. And then we had a game designer come in and play it, and they're like, "You know what? We just need to make this moving platform stop for a few seconds every time it hits something."

And it's like, yeah, great, that's great, but that's going to mess up the music really badly. Because then it'll get all out of sync and stuff, so it's just balancing that. Balancing that, and figuring out how to wrap our heads around that problem, has taken a while.

Sound Shapes

Do you try to match a shape to a sound that you think would match the shape?

JM: Is that a joke?

It's not a joke.

JM: Well, in Everyday Shooter I do that -- like, "What would this look like as a shape?" kind of thing. But here, our design process for entities is a little different. It's actually kind of still evolving. Do you want to talk about that? I don't even know what our process is.

Shaw-Han Liem: Yeah, you don't know. I don't think there's a literal one-to-one shape to sound, but we try to match, I think, behaviors to sounds, and the sort of overall visual element with what's happening sound-wise.

So when we're designing entities, me and Jon basically had to sit there and argue for a whole day about like, "I need this entity to do this," so that in the music editor you're going to be able to do cool music with it, and Jon's thing is, "I need the entity to do this other thing," because in the game it's going to make it really cool gameplay-wise if it has this behavior.

And so we have to spend a lot of time trying to find that overlap. What's the coolest part about both of those things -- where the music thing will be cool and the gameplay thing will be cool. And so when we find that overlap, that's how we know we've got something.

And simultaneously you're going for like a simpler visual aesthetic but also one where the shapes have to communicate what they're doing -- like whether you can stick to them, whether they're bouncy, or stuff like that.

JM: That's the main thing, right? When you look, it's not so much that the game itself looks cool, but it's when you see what's happening, like when you can see that, "Oh, this guy's making a high-hat sound" or "that stompy thing is making the beat," and, "these little things are making the notes," that's what looks really cool about the game.

In order to play the sounds, they're actually playing in a sequencer. There's an invisible bar that comes through and plays the sounds after you've activated them, which make a lot of sense in terms of making music sound good.

But in terms of knowing right away that there's a one-to-one correlation between your actions and the sound changing, the the feedback is not quite as strong. Did you consider other ways of doing it?

SL: There's a bunch of things happening musically, but the basic underlying system is that this game has a sequencer built into it, right? So this game is basically music sequencing software, and it has a bunch of different interfaces to interact with that sequencer. So one of them is what you just talked about -- where the grid sequencer, or a drum machine, that goes left to right.

That kind of tool is really good for when you're doing something more deliberate, or you want to create a certain musical phrase, or you want a certain musical phrase to correspond with something visually. But there's other ways of interacting with that sequencer that are built into the game.

For example, these drum lasers here operate independently of the grid, right? So you can place them anywhere, and also change the sequence by the player interacting with them. There's another more direct way... There's another entity which is basically a blank one note sample, right?

It's a blank one note sampler that responds when the player touches it, so basically depending on your movement, every time you play it, you're going to get a different phrase, because it's based directly on how you interact with it.

So these things will record one note based on when you hit them. Basically, if I recorded a phrase into these things, into these that was directly timed with what I did, now I go into this beat, these drum beats are added on top of that. And then I can also add those notes on top of it, so they're sort of like there's one master sequencer underneath that's controlling all the musical logic, and there's all these different ways of adding music to it. The notes are just one of the ways.

You know how people do those ridiculous Super Mario World sequences where everyone just straight bouncing through the world and doing a song? You could do that.

SL: We're looking forward to seeing what people make. I think that would be super awesome -- where you have to beat this level at a certain pace in order to keep up with the song. That's pretty hardcore, because then you have to do all the platforming.

But we like the fact that you can make sort of slow exploratory levels where the music sort of evolves at your pace, but you can also make more hardcore things where you're trying to keep the pace of the song and keep going.

How did you personally stay solvent for four years of not releasing a game?

JM: I live pretty cheaply, and Everyday Shooter was good enough to tide us over. Actually, no, wait. That's not true. We got a grant from the government. That's how I got him to quit his job.

And then Sony picked it up a year after that; after the grant ran out, we're like, "Oh, shit!" That makes sense. Because Everyday Shooter didn't do that good.

Do you think it's going to be another four years until your next thing?

JM: I don't know. I guess we'll find out. I don't want to do a crazy game, though; I want to do something simple. At least coming out of this, I want to do something simple.

It's funny, because visually it looks like a simple game. So people could perhaps have the mistaken impression that is a simple game.

JM: This is the thing. I can't do my crazy graphics shit, because then information gets lost, you know what I mean? Information is core to the visual. So in Everyday Shooter, all you needed to know was "don't touch stuff," right? But in here, you have to know some stuff you can stick to, some stuff you can stand on. Just it being a platformer makes it a lot harder.

And also because there's an editor. A lot of the stuff in Everyday Shooter is actually composed. It's not actually random, you know what I mean? It's like you know it will happen at these specific times. It's orchestrated, to an extent. But with the level editor I have no control over that, so I have to be super careful about what I'm going to do visually.

Everyday Shooter

Did you build everything, all the single player levels with the same editor that provide to people?

JM: We used our shitty PC editor, which is more fully featured, as sort of our test bed But it's really shitty. It's like on Vita, you can just select stuff, but on PC, you get to a point where you're just trying to select just one thing and it doesn't work.

So we're actually really -- and I know this sounds like PR bullshit -- but I'm super excited to get our Vita editor fully featured, so we can start using it instead of the PC one. And you can use the back touch. So you can look at the level while you're making it. It's a lot better.