Postcard from Siggraph 2005: Beyond the Gamepad

At a SIGGRAPH 2005's session devoted entirely to the game industry, a panel of notable developers discussed the future of gaming, focusing specifically on third-party peripherals and the world beyond the hand-held game controller.

Interfaces have always been an important part of gaming, said moderator Alex Pham of the LA Times, from the one-button Atari joystick to the complex driving wheels, bongo drums, and dance pads of today's market. With games like Karaoke Revolution and Gran Turismo 4 leading more and more people to purchase hardware accessories that enhance their gameplay experience, one begins to wonder just what the future of game controllers might look like.

The first half of the session consisted of game and controller demonstrations by the various speakers; the second half of the session was a moderated Q&A.

PART 1: Demos

Electronic Arts

Henry LaBounta and Habib Zargarpur, senior developers at EA with backgrounds in live action special effects, kicked off the demos by showing off the potential of next generation consoles.

EA's Fight Night title for the PS3 led the fray, with LaBounta showing real-time rendered in-game footage which, in HD and at 720p, looked almost pre-rendered. The detail was terrific - as one boxer struck the other, a slo-mo close-up showed a real-time "ripple" moving through the boxer's cheek, and dynamic sweat particles flying from the accurately-modeled pores on his face This was achieved using EA's proprietary facial texture capture system.

Zargarpur next showed off Need for Speed: Most Wanted, for Xbox 360, again in HD at 720p. Real-time views of the car racing action easily looked as good as pre-rendered CG from earlier systems, and an in-game fly-though showed off developer features such as dynamic time of day, real-time lighting and shadows, real-time particle effects, and other next-gen candy that wowed the crowd.

Harmonix Music Systems

The first peripheral-based demo came from Greg LoPiccolo of Harmonix Music Systems, the studio behind rhythm games Frequency and Amplitude. The game is called Guitar Hero, a guitar-simulation music game, and it's played with a plastic guitar, complete with five "frets," a strum bar, and a warbler.

The game is essentially an American re-imagining of Japanese title Guitar Freaks, in which the player hits notes in time with the game's musical and visual cues. However, Harmonix has changed things around, adopting the look of previous Harmonix games, so that notes come from space toward camera, rather than moving vertically on screen. This approach left room for an on-screen avatar to rock the virtual stage as LoPiccolo jammed to "Ziggy Stardust."

Sony Computer Entertainment America

The most interesting of the demos came from Richard Marks, Manager of Special Projects at Sony Computer Entertainment America, and involved his increasingly popular EyeToy.

The first demo highlighted a "View Tracking" system that allows the EyeToy to adjust the in-game camera based on movements of the player's head in space. Marks played a first-person shooter for the audience, and when his character came to a corner, he was able to "duck" his head around for a brief peek at his opponent. View Tracking uses about 25% of the CPU power of a PS2, Marks said - but less than 1% of the power of the PS3.

The second demo featured a gestural interface in which the EyeToy tracks and translates a user's hand motions, based on simple color keying. Holding a simple wedge of neon-colored Styrofoam in his hand, Marks was able to select and rotate objects in the game, effectively turning his hand into a spatial mouse. In addition to eliminating the need for gloves or other devices, he noted that this allows the EyeToy to track multiple objects at once, allowing for two-handed gestural control and other gameplay innovations.

Logitech

Logitech's Fred Swan, responsible for designing and marketing the company's game controllers, next showed off Logitech's GT Driving Force Pro Steering Wheel, designed specifically for use with Gran Turismo 4. As he operated the physically accurate wheel - engineered to 900 degrees of rotation, like a real steering wheel - and accompanying foot pedals, he noted that Logitech had worked closely with Polyphony (maker of GT4) for over two years to develop a controller that would be truly form-fitted to the game. He added that using the wheel to play GT4 not only makes the game more fun and intuitive, but also adds an element of realism that is impossible to achieve with a standard dual-shock controller.

Konami

Lastly, Michael McHale, Senior Producer at Konami Digital Entertainment, showed off a number of Konami's forthcoming rhythm games, including the next installments in the Karaoke Revolution and Dance Dance Revolution franchises.

McHale first showed off the forthcoming U.S. release of Beatmania, the popular Japanese DJ Simulation game in which players match sound samples to the beat per visual cues, via a unique turntable-and-keys controller.

He next showed off DDR Extreme 2, the fourth DDR title for the PS2, played via the classic four-arrow dance pad with the optional addition of the EyeToy. The dance pad is unique, Marks said, in that it functions for both entertainment and exercise, via the game's increasingly popular "Diet Mode." (He noted that DDR is currently being brought into schools in West Virginia as part of an obesity management program.)

Lastly, McHale ran a demo of Karaoke Revolution Party, a follow-up to previous Karaoke Revolution games in which the pitch of a player's singing is judged via a peripheral microphone. For the grand finale, McHale showed a new game mode in which the microphone, the dance pad, and EyeToy are all used simultaneously, allowing a player to dance and sing while appearing on enormous TV screens within the game.

Part II: Panel Discussion

After the demos, moderator Pham kicked off the discussion portion of the event with a series of questions for the panel.

The questions and responses below have been paraphrased for readability.

Mod: Let's get started. What makes a great game controller?

McHale (Konami): Well, it's good to have a standard controller, like the dual shock controller or the Xbox controller. Alternate controllers like the microphone or dance pad certainly have their place though. These tend to be more intuitive for people to use, and have alternate functions too, such as exercise. So I guess a well-designed controller is something that has many uses, or that provides new ways to interact with an existing game.

Swan (Logitech): Like Mike said, good controllers are intuitive. The steering wheel is a good example. People are familiar with this interface - no explanation needed. A really good game controller resonates with the user right off the shelf - they see it and know what it's gonna be used for. A great controller transforms a game experience to something really compelling and immersive.

LoPiccolo (Harmonix): It's also worth drawing a distinction between general purpose abstract controllers, like the dual shock controller, and specific-use controllers such as the guitar. Guitar Hero feels great when you play it with this plastic guitar, but the same game feels lame when you play it with the dual-shock. So in some regards it depends on the game.

Marks (SCEA): What's interesting is that there are some interfaces that kids use all by themselves, regardless of whether there's a game attached - like magic wands and light sabers. So how much cooler would it be if you could hook that light saber up to a video game and actually become a Jedi knight.

Mod: What are some of the risks and benefits of releasing a game that requires its own special controller?

Marks (SCEA): The big risk is that it adds cost to the game. A camera or microphone is something people can see as being repeatedly useful. but with more unique peripherals, people wonder if the controller will be useful for anything beyond the single experience, and this might make them hesitant to buy either the controller or the game. An idea like "Digi-Scents" [a recent scent-releasing interface] is interesting, but very niche and ultimately a riskier product.

LaBounta (EA): It's true, big companies like EA want to develop products that can work for multiple games, and to be able to develop multiple titles to match the product.

Swan (Logitech): Remember that you generally have to enter into a relationship with another company to do a hardware/software combination product. Hardware development involves issues that differ from software development - longer lead times, forecasting issues, different kinds of business relationships, and so on.

McHale (Konami): You do have to go into a niche project like that carefully - with lots of market testing, and so on. You don't want to end up with a warehouse full of extra peripherals. But you also have to stick with it once you commit to the idea. We shipped only 10,000 dance pads for the original PlayStation, but over time we've developed an install base of several million pads. As a result, it's now much easier to sell the DDR software.

Mod: Let's talk about the PS2's dual-shock controller - the best-selling game peripheral of all time. What are some of its strengths and weaknesses?

McHale (Konami): It's certainly a complicated controller compared to, say, the one-button joystick of the Atari 2600. Complexity might be a necessity for today's games, which themselves are quite complicated. but the learning curve of something like the dual shock can be a barrier of entry for many people. So, we need to have easy-entry controllers for casual players, and more complex controllers for the hard-core gamers.

Swan (Logitech): The dual shock controller is like any technology, which takes time to penetrate the culture and eventually become common. People who use computer keyboards from early in life tend to be natural typists as adults. The same type of physical memory develops with game controllers. In this respect, standardization is very important - it's important to stay consistent over time.

LoPiccolo (Harmonix): The dual-shock controller does have a reputation for being intimidating, but mostly to older people who don't play games. So once the old folks die off, everyone remaining should feel comfortable with dual shock style controllers! In terms of drawbacks - we need a standardized way to do easy text editing with game controllers.

Zargarpur (EA): One unexpected benefit of the dual-shock is the degree to which developers themselves can use it to do their work. [He loads up a demo of "ICE," the in-camera editor used in Need for Speed: Most Wanted. He then uses the dual shock controller to create an in-game cinematic, by first playing the game, then picking a series of cameras, cut points, and real-time lighting set-ups.] With these two sticks, this controller is almost more appropriate for 3D work than a mouse and keyboard.

Mod: What do you think are the key components of a next-generation gaming experience?

LaBounta (EA): Displays are going to be critical. I'd love to see a truly immersive 3D display, capable of giving a player a virtual reality experience without all the messy goggles and wires. HD-R displays are really cool too - I love the idea of looking at a TV and feeling like you're actually looking out a window upon a real scene. And ultimately, anything you can do to get people off their couch and get them active has to be good!

Marks (SCEA): Well at Sony we're working on taking the EyeToy into 3-dimensions..

[At this point he narrated the audience through another demo, in which he used hand gestures to cast magical spells via a dimensional "force field" - a virtual plane located about one foot in front of the player. Only the person crossing the threshold of the plane was able to cast magic - a second person standing in the background was not recognized by the camera.]

[He continued the EyeToy demo by showing an idea for a virtual potter's wheel, in which the user rotated their hands around an imaginary lathe to create 3D pottery. In a third demo, he showed how the console could move a group of butterflies both in front of and behind the image of a player, tracking the position and occlusion of a colored ball.]

[Lastly, he showed a Matrix-style bullet-dodging game, in which the player physically leaned in various directions to avoid bullets coming at his on-screen character.]

Mod: OK. How about voice interfaces? Will we be seeing these in games anytime soon?

Swan (Logitech): Well, we're on our way. It started with SOCOM. But the concept of creating a world where the game understands everything you say and where every character has enough AI to be able to respond intelligently is a huge challenge. Especially when you consider the sheer length and open-ended nature of the average game.

Marks (SCEA): Voice recognition, from a technological standpoint, is not that hard. We'll be able to understand phoneme strings very soon. It's understanding meaning and context that's really the challenge.

McHale (Konami): Yeah, there are a lot of issues. People have different accents, different dialects. and players get frustrated when the game doesn't understand what they're trying to say.

LaBounta (EA): It can be daunting just to localize menu systems. think what a huge undertaking it would be to do this with spoken language!

Zargarpur (EA): And then magnify the problem again by all the complications of huge online games. we've got a ways to go on this one.

Mod: Last question. We're at the world's biggest graphics conference, but let me ask you this: are graphics really important to games?

McHale (Konami): Games are looking great today, and games certainly have to have great visuals to be successful in this era, but good gameplay is what matters the most.

LaBounta (EA): Gameplay is of course important, but an immersive experience includes strong graphics. Games actually still look quite bad in my opinion - we have a lot of room to improve in terms of cinematic quality. One reason people loved SSX3 was because of our successful combination of good gameplay and beautiful visuals.

McHale (Konami): You're right, you do have to deliver the full package. The game industry is learning a lot from the film industry right now, especially in terms of cinematography, lighting..

LoPiccolo (Harmonix): Yes. And as our graphical tools continue to converge, game developers will increasingly be able to use intuitive, professional animation tools to create great-looking game graphics with a more reasonable amount of work.

Zargarpur (EA): Graphics-wise, it doesn't take a lot for your mind to get sucked into a game - gameplay is king, and Pong is the proof. But as graphics get more and more advanced, our minds get further immersed in the experience. Still, as Greg says, it will be great when we can stop thinking quite so much about graphics and move back into thinking about gameplay.

Mod: Thank you all for your time.

______________________________________________________