The Saturday Paper - Words, Sounds and Pegasi

Originally posted on Games by ANGELINA

One of the most wonderful things about videogames is how they bring together different mediums into one collective effort. Art, music, writing, design as well as programming itself are all hugely creative endeavours in their own right, and often we see all of these and more in a single game. The Saturday Papers mostly focuses on design and programming, but today we're looking at three papers from The International Conference  on Computational Creativity, which was held a few weeks go in Sydney, Australia. Can computers help us do more than just generate levels?

More...

The first paper is Once More, With Feeling! Using Creative Affective Metaphors to Express Information Needs by Tony Veale. Tony is behind the suite of wonderful linguistic tools over at the University of Dublin's Creative Language System Group, and this year he presented Metaphor Magnet, a seemingly simple tool that hides a multitude of really cool functionality. Metaphor Magnet has built up a huge database of linguistic information using Google N-Grams - not the book database that Google supplies online, but a more detailed corpus of phrases Google has found on the Internet.

Metaphor Magnet allows you to ask for a metaphorical relationship between two things, such as love is a drug. It then looks into the corpus for properties that people ascribe to these two ideas, and tries to make connections between them. What you get back are pairs of adjectives and nouns, like "raging:disease". You can click on these relationships to get more information on them - Metaphor Magnet can give you evidence from the corpus for each association, or it can give you other concepts with similar properties. You can skew the metaphor to be positive or negative using symbols too, so do you want love as a +drug or love as a -drug? You can even ask it for emotions - like 'pained' - and it will tell you things that people claim to be pained by.

The best part of Metaphor Magnet is that it provides an XML web service through which anyone can make requests of the service and get back linguistic information of all kinds. I wrote a quick Gist in Python which takes an RPG occupation (like thieves) and a patron animal (like wolves) and produces hero titles that try to convey that animal. I can see all sorts of interesting poetic uses of the tool, and Tony is keen to see what people can come up with too. Let him know if you use the tool anywhere!

The second paper I want to talk about is Computationally Created Soundscapes with Audio Metaphor by Miles Thorogood and Phillipe Pasquier. This is a beautiful paper describing a very cool tool, but it also has some other stuff hidden under the surface that could have big implications in the future, too! The tool that the paper describes can take sentences describing a location or an ambience, and then try to compose a soundscape using sound effects that describe the area. For instance, you might give the system:

a quenching rain drenched my burning head

From which the system can go online to find sound effects and build them into a piece of audio. The way in which it does this is incredibly nuanced - the tool uses Freesound to get the initial set of sounds, but Miles and Phillipe developed the software to be able to differentiate between foreground and background noise. It can do this with really great accuracy too, and this gives it the ability to produce very interesting pieces, using thoughtful composition rules to decide when to include foreground detail, and when to shift and vary the background noises. There are some great samples on their site here.

Miles and Phillipe mentioned videogames in their talk as a specific application area here. For the experimental readers, using Freesound is a great way to get unusual compositions from almost any input string. But they were also optimistic about the use of the tool in AAA productions too, using a professional sound database tagged appropriately. Many games generate their soundscapes by just playing sounds based on whatever is nearby - animals, machinery, and so on. This system is far more intelligent, and can produce some really great compositions from just a simple description of the environment it's in.

Finally, Using Theory Formation Techniques for the Invention of Fictional Concepts, by my CCG colleagues Flaminia Cavallo, Alison Pease, Jeremy Gow and Simon Colton. I've wanted to talk about this work for a while and this paper is a nice illustration of where they're going with it. AI is often used to reason about things that are real and already exist, but doesn't often think about fictional concepts or things that might not be possible in reality. Of course, this is what games are often about, so a paper looking at this is definitely of interest!

Flaminia and co. used some specialised AI tools to look at a database of existing animal concepts annotated with facts we know about them - things like the fact that a penguin is a bird, or that it swims. Then they started considering what you'd need to create fictional animals, but in particular fictional animals that are interesting in some way. Lots of games have random generation for things like creatures - Starbound is selling itself on the basis that you might find anything on the next planet you land on. But 'anything' is quite a boring concept. You might find exactly the same horse you saw on the last planet, but coloured blue. That's certainly random, but it's not that cool.

This paper shows the start of a new line of research that moves away from that. It's not just concerned about generating an animal that isn't in the database already. They're interested in animals that are surprising, or interesting, or thought-provoking in some way, and they've tried to describe what 'interesting' means in terms of what knowledge about the world your animal-inventor already has. While the system itself isn't easily integrated into a game because of the software it uses, I wanted to give this paper a special mention because it might inspire people who are randomly generating content for their game, and make them think about ways they can filter their content to make it a little more interesting and a little less random.

A special mention goes to the Automatic Composition of Lyrical Songs paper which I didn't include here fully because it's not totally related to games in its current state, but the thought of being able to custom-compose a theme song for the player is too cool to not include. The first RPG that can invent a theme song for me based on my heroic actions will win my lasting adoration.

Where To Find More

All these papers and more are available on the ICCC website here, and you can find papers from previous years there too. Contact the authors, get in touch, and tell them you want to use their ideas and software! Scientists need a push to put their stuff out there, I think, and the knowledge that people are excited by their work is a great motivator. I'd love to see some of this stuff in games.

I hope you enjoyed this departure from the usual Saturday Paper themes! Let me know what you think over on Twitter, and normal service will resume next time.

Submit your papers to our AIIDE Workshop!

To close, another friendly reminder that July 3rd is the deadline for papers to AI & Game Aesthetics, a workshop at AIIDE 2013. We're inviting short papers, of up to four pages, from academics, game developers and more, on a host of subjects relating to AI and subjective issues in game design. More details here, email me if you have any questions!