Microsoft's Tynes Helps Kids Kinect With Project Columbia Prototypes

John Scott Tynes, lead designer and head of IP partnership for Microsoft's Kids and Lifestyle Entertainment division, has been interested in computers since he first got one back in 1981. From there, he got into programming and writing his own games, but long maintained an interest in tabletop and strategy games as well, including twelve years in the tabletop space working with Wizards of the Coast on Magic: The Gathering. The first computer game he shipped was PC MMO Pirates of the Burning Sea, before joining Microsoft to work on XBLA projects. He's currently at work there on an unnamed Kinect title for kids codenamed Project Columbia. The work of game development often creates a number of complications that cross-disciplinary designers can overcome through expertise, and finding creative solutions to these problems is one of Tynes' interests. At the NYU Poly-Game Innovation Lab on Wednesday, Tynes discussed creative problem solving in the marriage between tech and design. He spoke about how Microsoft is currently working to address specific interface challenges in the development of Project Columbia - which combines the challenge of learning user behavior with the Kinect device with the problematic fact that the target audience is young children, who can neither read instructions or necessarily understand traditional inputs and must use their bodies to communicate with the device. Imagine a game like Rock Band, which comes with an enormous content library that requires a sophisticated navigation interface. Tynes' current project has storybooks, and will ship with somewhere between seven to fifteen storybooks on disc and sell others later as DLC. Now, imagine having to teach energetic, vocal kids - who additionally might speak too fast, energetically or not clearly enough to communicate via speech with the Kinect, let alone if they have friends along -- to navigate such a detailed content library without readable instructions or button presses. Tynes and his team have begun looking at the challenge of how to offer kids the opportunity to select storybooks under these significantly challenging circumstances. "Whatever instructions we need to give them need to be visual and verbal," says Tynes. On top of that, it's a Kinect game, which means inputs become more challenging as kids have to understand cues and then use their bodies without the use of traditional inputs and instructions that have worked for other Kinect games. Thus far the team has prototyped three solutions, says Tynes. "We use augmented reality extensively in this project, because kids fundamentally love seeing themselves on TV. It's also great because if you use Xbox and Kinect, the interfaces you fundamentally experience are based on things like, 'hold out your hand and a hand-eye cursor appears on screen.'" Direct relationship between touch and movement is immediately easy to understand, says Tynes, which is why kids love iPhones, iPads and other touch devices. With Kinect there's an additional layer of abstraction because it uses gesture - but using augmented reality closes that gap in addition to being a pleasure for children. One method the team has been studying places the kids in a middle of a waterfall of slowly-falling books, and they can reach out and grab one. Having the books act as physics objects that bounce off the child or stick to his or her arm helps wordlessly teach the child that their image can affect the reality in which the books exists. It may not the quickest and most efficient way for kids to seek and select books - but children don't care about efficiency in the way that adult users do. When working with kids it's more permissible to do a user interface that favors intuitive playfulness over efficiency. However, there are problems with the waterfall UI idea - what happens if a kid touches a book with his or her hand by accident? It's harder to communicate and confirm selections with this interface. So another choice for the team is its Carousel prototype, where the child is on screen below a rotating "carousel" of books that drift in one direction on a sort of ring in space. Kids can reach up and grab the book they want as it passes, and more advanced users who play and experiment often can learn to affect the speed and direction of the carousel, as well as stop and start it. That creates opportunities within the interface for experimentation and learning. Third is the "hula hoop" or "utility belt" model, where the ring of books is attached to the user's waist. "The good thing about this model is these books are kind of floating around you, and the whole thing moves with you." That communicates the reality and tangibility of the game world in a way the floating carousel doesn't. The team is still experimenting with interfaces and hasn't finalized its ideas. In all cases the team does prototyping and usability testing with kids brought into Microsoft's labs so that the team can see how they interact with it and what their emotional reactions are. That type of problem-solving inclination should be in any developer's toolkit, he recommends: Those who don't want to get under the hood of a given infrastructure and learn how various elements work should probably choose another line of work, he says, characterizing game design as an "extremely integrated" profession requiring a wide range of vocabulary and understanding. A healthy appetite for tech and design problem-solving should be in any developer's toolkit, Tynes recommends: Those who don't want to get under the hood of a given infrastructure and learn how various elements work should probably choose another line of work, he says, characterizing game design as an "extremely integrated" profession requiring a wide range of vocabulary and understanding.