The Designer's Notebook: Hardware Designers - Talk To Us!

Back in 1992, about three months after I started at Electronic Arts, Rich Hilleman, my producer, walked into my cubicle. "Ernest," he said without preamble, "is your passport up-to-date?"

"Uh… yeah, always," I said. "Why?"

"How would you like to go to Japan?"

"Okay… when?" I said, wondering if I was going to be on a plane that evening. I had never been to the Far East before.

"Next week. I want you to attend a training session on the new Sega Genesis CD add-on."

"Rich, I've never programmed the Genesis." I was hired as a PC programmer. I didn't tell him that I'd never written a program in assembly language, much less Motorola 68000 assembly language, in my life. I had intentionally stayed out of the game industry until it was possible to use high-level languages. All I had ever done was some optimization and bug-hunting through the PC's 8086 machine code.

"That's OK. Talk to Kevin McGrath, he'll tell you all about it." Ah, the cheerful complacency of the non-techie.

So Kevin McGrath, a Genesis god at EA, gave me a crash course in its hardware and programming, and, thank goodness, he went with me as well. In a week we were in Japan, the guests of Sega, learning how the new CD add-on device was going to work. It was more than just a CD drive, it turned out; it included its own RAM and an additional 68000 microprocessor. That'll open a few developers' eyes, I thought - most game programmers at the time were self-taught and knew little computer science, much less the intricacies of parallel processing and deadlock prevention. But the Sega people seemed to be avoiding one particular issue, so on about the third day there, I stuck up my hand.

"How does the program know when a data block is available from the CD drive?" I asked.

"You have to check the status byte," they said.

"You mean, poll it?"

"Yes."

"It doesn't raise a hardware interrupt when the data's ready?"

"We are pleased to say that you may check the status byte to see if new data are available," said the Japanese Sega employees. "No," said the one American Sega employee.

Great. Here we have about the slowest storage device since punched cards, and you have to continually ask it whether the next block of data has arrived. 99% of the time it hasn't, of course. What a colossal waste of CPU time. As the class went on, it turned out that several other things had to be polled as well. I couldn't believe it. Sega had saved themselves a few thousand dollars' worth of hardware engineering time, plus maybe a penny or so per unit, by not including a hardware interrupt line from the CD drive. And we programmers, and every game we made for the wretched thing, were stuck with the consequences.

Now I know that console manufacturers have to squeeze every nickel they can out of the hardware to keep the price down. It's a consumer entertainment device, not a PC, after all. Still, I do sometimes wish they would talk to some game developers before they designed their machines.

There's a rumor going around that the Playstation 3 will be "1000" times as fast as the Playstation 2, thanks to their new cell architecture. But I have to ask myself what this means in real terms. Will the games be 1000 times as entertaining? 1000 times as smart? Did any game programmer come to them and beg for a machine 1000 times as fast? I doubt it. Given the choice, I'd rather it were 10 times as fast and had 100 times as much RAM. Or it were 1000 times as cheap. Can you imagine how big our markets would be if a Playstation cost 30 cents? Don't laugh; you can buy watches and pocket calculators out of gumball machines, and simple personal organizers are getting close.

It seems to me that raw CPU horsepower is not really our biggest need at the moment. But hardware designers design computer hardware in order to maximize processing power, because that is their area of interest and expertise, that's what they've been trained for. Even though they are designing a machine explicitly intended for playing games, for offering entertainment experiences, they still treat it primarily as a data-processing device. Computers were invented for calculating ballistics tables for artillery shells, and in essence that is what hardware designers still optimize them to do. There is a distinct disconnect between the intended purpose of the device and its designers.

On the other side of the equation, game designers and programmers get handed a new machine without ever being consulted about its capabilities. It's simply given to us, and our approach is, "Well, let's see what can be done with this thing." As a game designer, I wish that somebody would invent a chip that did pathfinding. But I'm at the wrong end of the development chain; I never meet any hardware designers. I'd like to.

I wonder if the designers of the Playstation 3 have really taken a close look at what game programmers are spending their time on these days. Have they talked to each other? Since I'm not privy to Sony's inner workings, I have no idea. But it seemed clear to me that the designers of the Sega CD add-on didn't talk to (or, rather, didn't listen to) any game programmers back in 1992. And it may be no coincidence that Sega doesn't have any hardware designers any more.

So what are we doing in software at the moment, that we might be doing in hardware instead? Well, pathfinding, of course, but pathfinding is harder to generalize to a hardware solution than 3D graphics are. Rendering textured triangles onto a viewing plane is pretty universal. Pathfinding is needed by a smaller subset of the game world and its parameters are much more variable. What else might there be?

At this year's Game Developers' Conference in Europe, Jason Rubin opined that games have now reached a point at which graphics are no longer our primary selling point, and very soon we will have to concentrate on new things to attract the consumer's attention. We can do photorealism without difficulty, so 3D games are finally starting to branch out a bit and examine other art styles. Jet Grind Radio, Viewtiful Joe, and the forthcoming XIII from Ubi Soft are good examples, as is Pencil Whipped, an indie first-person shooter set in an entirely pencil-drawn environment with pencil-drawn baddies. (Pencil Whipped would be a better game if it didn't contain a Twinkie Denial Condition: it speeds up on a fast machine.)

I agree with Jason that we've reached a point of diminishing financial returns on increasing graphic quality, at least as far as static models are concerned. But we're certainly not there with animation. We've now got amazingly-detailed people... who still look like a Disney animatronic the instant they start to move. Nvidia's leaf-clad fairy named Dawn, a technology demo intended to show off their wonderful ability to render the human form, still walks with a strange, mincing motion. Her twin, Dusk, dances no better than I do, which is a pretty severe condemnation. If they're supposed to be the ne plus ultra of animation, it only demonstrates how much farther we have to go, and I think hardware may be part of the answer.

(I realize at this point, a few of you are probably gasping, "Can this be the author of Dogma 2001 talking? I thought you hated hardware accelerators!" My reply is, no, I don't hate hardware accelerators. I hate treating hardware as if it were a source of creative inspiration, a reason for building games, rather than a tool for making games look and sound and play better. What I don't like is people building a glitzy technology demo with zero gameplay innovation, and hyping it as the greatest game in the universe - which, frankly, amounts to cheating their customers. Such products are the game equivalent of those corny 3D movies made in the 60s: interesting technology, no plot.)

Like facial expressions and tones of voice, movement is one of those things we understand at a very deep level, without even thinking about it. If you've ever noticed the strange wooden way that that actors sometimes walk in amateur theater, you're observing that sense in action. Just the knowledge that other people are watching you walk can make your movements awkward and unnatural. Serious actors spend hours working on the walks of the characters they're going to portray, so that when they're being watched by a theater full of people - or a whole movie crew - they can still produce a natural motion that is also appropriate for the character.

One of our problems is that we use simple walk cycles which endlessly repeat. People don't actually walk in a mathematically uniform rhythm unless they're marching on parade. Years ago in this column ("Not Just Another Scary Face") I suggested that we should vary the appearance of our people and creatures by making small random changes to the mesh and textures; now I'd like to propose that we do the same for walk, run, climb, and other animation cycles by making small random changes to their kinematics. The changes have to be subtle, though, or the results will look goofy, like something out of the Ministry of Silly Walks. They'll also have to balance out to zero: a slightly shorter stride on one step will need to be compensated for by a slightly longer one on the next step or two, or you might get a series of shorter and shorter strides just through an odd run of luck. As with deforming meshes, it would be a good idea to generate Gaussian, rather than equally-distributed, random numbers.

Inverse kinematics will help a lot with strange-looking animations. At the moment it's quite common in games to see people whose feet sink into the ground when they're walking up a slope, or whose stride up a staircase doesn't actually match the pitch of the stairs. A fixed stair-climbing animation cycle doesn't allow for different kinds of staircases. With inverse kinematics, we calculate the position of the foot not by working outwards from the character's torso, but by working backwards from the stair he's stepping onto. Of course, this has to be done within natural limits, and the position of his torso adjusted appropriately to account for his shifting weight - people lean forward more when they're walking up steep staircases, to put more of their weight directly above the foot in front.

IK is processor-intensive, since every step and other movement has to be computed on the fly based on the environment around the character, rather than simply displaying a standard cycle. True locomotion, the next step beyond IK, is even more processor-intensive. It involves simulating the masses of the character's body parts and the forces exerted on them as he walks, to simulate complete body movement on the fly. At the moment, this technology is mostly used in situations where it can be done in advance: pre-rendered CG movies rather than video games.

But perhaps if there were some specialized animation hardware… is anybody listening?