Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs or learn how to Submit Your Own Blog Post
For over 4 months in my spare time, I've been working on an artificial intelligence named Piglet that plays and experiments with classic gameboy color games. In a series of articles, I explain what that process has looked like and how Piglet works.
In this series of articles, I discuss the origins of Piglet, how she's worked at various points in her development, and what some of the design philosophies that went into her development were. If you're interested in learning more about the current version Piglet, check out her site at http://danshumway.github.io/Piglet/. Piglet is currently being rewritten to run in a NodeJS environment and will be relaunching soon!
Imagine you're blindfolded, and handed a controller, and told to play a game. Every time you press a button something happens in the game, but you can't see that happening. Your only interface with the game is that you have a friend sitting next to you - and whenever something on the screen changes, he shouts in your ear, "Hey, something changed!"
It works surprisingly well.
Piglet is a machine-learning based AI that tries to play classic gameboy and gameboy color games using a variety of techniques. She's built using Lua, plays her games using the Visual Boy Advance platform, and for the past month I've slowly starting to stream everything she does on Twitch.tv. Oh, and she's open source, so you can see all of her code and guts on Github.
I originally started work on Piglet under a different name, WideEyes, at a hackathon back in early February of 2014. At the time, I was just starting to dig deeper into a couple of different scripting languages, and I was looking for an excuse to start actually working with APIs and other applications. It was really bothering me that I was digging very deep into how specific languages and data structures worked, but not how to actually leverage them to do any cool things with other existing applications or programs or services.
I had also been following a couple of different projects that were ridiculously fascinating to me, including Playfun (see below), and Twitch Plays Pokemon. In particular, I was really intrigued by the way the Tom created a personality around Playfun and humanized its reactions.
I wanted a piece of that cake. Well. I wanted a different cake that shared similar qualities, but branched out in significant ways, like it had chocolate frosting instead of vanilla, and it was a different shape because I used a different pan to bake it.
Playfun had some weaknesses. It wasn't real time, and it was primarily concerned with finding goals, not learning the rules of the game itself. If I was was creating an AI, I would want my agent to form an idea of how to play optimally, not just what optimal play looked like.
What those concerns look like in a practical sense when building an AI are listed below.
Coming up with goals shouldn't be the focus or most complicated part of the program. Accomplishing those goals reliably is.
Piglet should be able to run in real time (as opposed to executing a pre-determined strategy, similarly to something like Playfun.)
Piglet should fulfill a particular telos when she runs of experimentation and "proper" play - ie. Piglet should have underlying philosophies of how she plays games. This one is a bit more tricky, but it does have a lot of application to how Piglet's code is laid out and what algorithms I choose.
The AI I decided to build was named WideEyes, and the way I decided to approach building it was by modeling how (I thought) humans played games, at least in a very simplistic manner. WideEye's goal would be to avoid boredom.
Visual Boy Advance will return around 65,000 individual bytes - not actually very much information. I had never worked with reading RAM before, so I knew my algorithm needed to be pretty simple. My goal was to get Piglet moving on the screen in an interesting way, everything else was secondary to me getting started with Lua, learning how to interact with Visual Boy Advance, and so on.
The basic program loop I decided on is describe below, and you can see the actual code at Github.
Each frame, WideEyes loops through VBA's RAM and checks to see how many bytes have changed in value since the last frame. It returns the number of frames back to the brain. WideEyes treats this as an "interest value", representing how much fun it's having with the current game.
If a certain number of frames have elapsed (60), WideEyes creates a savestate and stores it in a list.
If a certain number of frames have elapsed (30), WideEyes checks to see if those frames have passed a certain threshold of interest. If they have, awesome. If they haven't WideEyes resets the game to the last savestate and then pops it out of the list of saved savestates. This lets WideEyes keep on backing up if it really gets stuck or bored.
WideEyes can now choose its input. WideEyes chooses its input based on what worked well in the past (it has a very memory of 200 frames).
WideEyes loops through its memory of past inputs and results, and grabs the top 9 values. WideEyes purposely uses a broken algorithm for picking the top 9 values that occasionally gives it bad results. This increases the natural entropy of those results (more on this later).
If the combined "interest values" of the top 9 frames are not above WideEyes' threshold of interest, WideEyes just presses random keys.
Otherwise, WideEyes loops through the inputs returned from the previous loop and builds a key combo to press based on a scoring system that pits certain keys against each other. For example, up and down can't both be pressed at the same time, so whichever one appears most often in the 9 best frames is the key that's pressed.
Finally, WideEyes holds down those keys for an arbitrary amount of time (in case they have a delayed effect). I used 3 frames for this, but it could be just about any number. It seems to vary from game to game.
I was pretty happy with WideEyes at the time. It could do some interesting things with Mario. It produced slightly better than brownian motion (which was far bettter than anything I could have hoped for when I started the project), and it had enough (percieved) personality traits that it could demo nicely in small 5 minute presentations.
This is a stupid algorithm, and it shouldn't work as well as it does. There are a couple of reasons why it doesn't conform to expectations.
This approach allows WideEyes to rapidly change its strategy whenever necessary. If WideEyes runs into a wall, it almost instantly changes its strategy.
There's a constant high variance in what buttons WideEyes presses. I mention this a couple of times above, but a few of WideEyes algorithms are purposefully broken in ways that increase entropy over time. This introduces a natural decay into WideEye's memory that works pretty well to keep information changing in interesting ways.
Because of this WideEyes is almost constantly forming patterns of some sort, which fits that random data into something that looks more deliberate than it is.
WideEyes keeps track of its progress and doesn't reset back to the beginning of a level when it dies (it can savestate anywhere and correct its mistakes).
All of this works together to make WideEyes very good at game genres like Mario, specifically because Mario is using a lot of repetitive motion, and usually the way to make progress can be summed up with very simple formulas that change quickly. You can make a lot of headway by just holding down right and jumping randomly (although you'll die a lot).
Similarly, in Mario, these strategies are usually based on very recent actions - you die because you ran into something a couple frames ago, not because you made a poor move 30 seconds ago. Different games give WideEyes more trouble (it could never play something like Pokemon), but I was able to essentially throw only games at WideEyes that I knew it could handle.
When Piglet was born and I started to redefine how WideEyes worked, I threw out a lot of this math, and a lot of existing performance, because these systems are only deceptively good. In order for Piglet to make serious progress on a game and become truly competent she'd need to drastically redefine how she looked at games.
In this series of articles, I discuss the origins of Piglet, how she's worked at various points in her development, and what some of the design philosophies that went into her development were. If you're interested in learning more about the current version Piglet, check out her site at http://danshumway.github.io/Piglet/. Piglet is currently being rewritten to run in a NodeJS environment and will be relaunching soon!
Imagine you're blindfolded, and handed a controller, and told to play a game. Every time you press a button something happens in the game, but you can't see that happening. Your only interface with the game is that you have a friend sitting next to you - and whenever something on the screen changes, he shouts in your ear, "Hey, something changed!"
It works surprisingly well.
Piglet is a machine-learning based AI that tries to play classic gameboy and gameboy color games using a variety of techniques. She's built using Lua, plays her games using the Visual Boy Advance platform, and for the past month I've slowly starting to stream everything she does on Twitch.tv. Oh, and she's open source, so you can see all of her code and guts on Github.
I originally started work on Piglet under a different name, WideEyes, at a hackathon back in early February of 2014. At the time, I was just starting to dig deeper into a couple of different scripting languages, and I was looking for an excuse to start actually working with APIs and other applications. It was really bothering me that I was digging very deep into how specific languages and data structures worked, but not how to actually leverage them to do any cool things with other existing applications or programs or services.
I had also been following a couple of different projects that were ridiculously fascinating to me, including Playfun (see below), and Twitch Plays Pokemon. In particular, I was really intrigued by the way the Tom created a personality around Playfun and humanized its reactions.
I wanted a piece of that cake. Well. I wanted a different cake that shared similar qualities, but branched out in significant ways, like it had chocolate frosting instead of vanilla, and it was a different shape because I used a different pan to bake it.
Playfun had some weaknesses. It wasn't real time, and it was primarily concerned with finding goals, not learning the rules of the game itself. If I was was creating an AI, I would want my agent to form an idea of how to play optimally, not just what optimal play looked like.
What those concerns look like in a practical sense when building an AI are listed below.
Coming up with goals shouldn't be the focus or most complicated part of the program. Accomplishing those goals reliably is.
Piglet should be able to run in real time (as opposed to executing a pre-determined strategy, similarly to something like Playfun.)
Piglet should fulfill a particular telos when she runs of experimentation and "proper" play - ie. Piglet should have underlying philosophies of how she plays games. This one is a bit more tricky, but it does have a lot of application to how Piglet's code is laid out and what algorithms I choose.
The AI I decided to build was named WideEyes, and the way I decided to approach building it was by modeling how (I thought) humans played games, at least in a very simplistic manner. WideEye's goal would be to avoid boredom.
Visual Boy Advance will return around 65,000 individual bytes - not actually very much information. I had never worked with reading RAM before, so I knew my algorithm needed to be pretty simple. My goal was to get Piglet moving on the screen in an interesting way, everything else was secondary to me getting started with Lua, learning how to interact with Visual Boy Advance, and so on.
The basic program loop I decided on is describe below, and you can see the actual code at Github.
Each frame, WideEyes loops through VBA's RAM and checks to see how many bytes have changed in value since the last frame. It returns the number of frames back to the brain. WideEyes treats this as an "interest value", representing how much fun it's having with the current game.
If a certain number of frames have elapsed (60), WideEyes creates a savestate and stores it in a list.
If a certain number of frames have elapsed (30), WideEyes checks to see if those frames have passed a certain threshold of interest. If they have, awesome. If they haven't WideEyes resets the game to the last savestate and then pops it out of the list of saved savestates. This lets WideEyes keep on backing up if it really gets stuck or bored.
WideEyes can now choose its input. WideEyes chooses its input based on what worked well in the past (it has a very memory of 200 frames).
WideEyes loops through its memory of past inputs and results, and grabs the top 9 values. WideEyes purposely uses a broken algorithm for picking the top 9 values that occasionally gives it bad results. This increases the natural entropy of those results (more on this later).
If the combined "interest values" of the top 9 frames are not above WideEyes' threshold of interest, WideEyes just presses random keys.
Otherwise, WideEyes loops through the inputs returned from the previous loop and builds a key combo to press based on a scoring system that pits certain keys against each other. For example, up and down can't both be pressed at the same time, so whichever one appears most often in the 9 best frames is the key that's pressed.
Finally, WideEyes holds down those keys for an arbitrary amount of time (in case they have a delayed effect). I used 3 frames for this, but it could be just about any number. It seems to vary from game to game.
I was pretty happy with WideEyes at the time. It could do some interesting things with Mario. It produced slightly better than brownian motion (which was far bettter than anything I could have hoped for when I started the project), and it had enough (percieved) personality traits that it could demo nicely in small 5 minute presentations.
This is a stupid algorithm, and it shouldn't work as well as it does. There are a couple of reasons why it doesn't conform to expectations.
This approach allows WideEyes to rapidly change its strategy whenever necessary. If WideEyes runs into a wall, it almost instantly changes its strategy.
There's a constant high variance in what buttons WideEyes presses. I mention this a couple of times above, but a few of WideEyes algorithms are purposefully broken in ways that increase entropy over time. This introduces a natural decay into WideEye's memory that works pretty well to keep information changing in interesting ways.
Because of this WideEyes is almost constantly forming patterns of some sort, which fits that random data into something that looks more deliberate than it is.
WideEyes keeps track of its progress and doesn't reset back to the beginning of a level when it dies (it can savestate anywhere and correct its mistakes).
All of this works together to make WideEyes very good at game genres like Mario, specifically because Mario is using a lot of repetitive motion, and usually the way to make progress can be summed up with very simple formulas that change quickly. You can make a lot of headway by just holding down right and jumping randomly (although you'll die a lot).
Similarly, in Mario, these strategies are usually based on very recent actions - you die because you ran into something a couple frames ago, not because you made a poor move 30 seconds ago. Different games give WideEyes more trouble (it could never play something like Pokemon), but I was able to essentially throw only games at WideEyes that I knew it could handle.
When Piglet was born and I started to redefine how WideEyes worked, I threw out a lot of this math, and a lot of existing performance, because these systems are only deceptively good. In order for Piglet to make serious progress on a game and become a truly competent learner she would need to drastically redefine how she looked at games.
In this series of articles, I discuss the origins of Piglet, how she's worked at various points in her development, and what some of the design philosophies that went into her development were. If you're interested in learning more about the current version Piglet, check out her site at http://danshumway.github.io/Piglet/. Piglet is currently being rewritten to run in a NodeJS environment and will be relaunching soon!
Imagine you're blindfolded, and handed a controller, and told to play a game. Every time you press a button something happens in the game, but you can't see that happening. Your only interface with the game is that you have a friend sitting next to you - and whenever something on the screen changes, he shouts in your ear, "Hey, something changed!"
It works surprisingly well.
Piglet is a machine-learning based AI that tries to play classic gameboy and gameboy color games using a variety of techniques. She's built using Lua, plays her games using the Visual Boy Advance platform, and for the past month I've slowly starting to stream everything she does on Twitch.tv. Oh, and she's open source, so you can see all of her code and guts on Github.
I originally started work on Piglet under a different name, WideEyes, at a hackathon back in early February of 2014. At the time, I was just starting to dig deeper into a couple of different scripting languages, and I was looking for an excuse to start actually working with APIs and other applications. It was really bothering me that I was digging very deep into how specific languages and data structures worked, but not how to actually leverage them to do any cool things with other existing applications or programs or services.
I had also been following a couple of different projects that were ridiculously fascinating to me, including Playfun (see below), and Twitch Plays Pokemon. In particular, I was really intrigued by the way the Tom created a personality around Playfun and humanized its reactions.
I wanted a piece of that cake. Well. I wanted a different cake that shared similar qualities, but branched out in significant ways, like it had chocolate frosting instead of vanilla, and it was a different shape because I used a different pan to bake it.
Playfun had some weaknesses. It wasn't real time, and it was primarily concerned with finding goals, not learning the rules of the game itself. If I was was creating an AI, I would want my agent to form an idea of how to play optimally, not just what optimal play looked like.
What those concerns look like in a practical sense when building an AI are listed below.
Coming up with goals shouldn't be the focus or most complicated part of the program. Accomplishing those goals reliably is.
Piglet should be able to run in real time (as opposed to executing a pre-determined strategy, similarly to something like Playfun.)
Piglet should fulfill a particular telos when she runs of experimentation and "proper" play - ie. Piglet should have underlying philosophies of how she plays games. This one is a bit more tricky, but it does have a lot of application to how Piglet's code is laid out and what algorithms I choose.
The AI I decided to build was named WideEyes, and the way I decided to approach building it was by modeling how (I thought) humans played games, at least in a very simplistic manner. WideEye's goal would be to avoid boredom.
Visual Boy Advance will return around 65,000 individual bytes - not actually very much information. I had never worked with reading RAM before, so I knew my algorithm needed to be pretty simple. My goal was to get Piglet moving on the screen in an interesting way, everything else was secondary to me getting started with Lua, learning how to interact with Visual Boy Advance, and so on.
The basic program loop I decided on is describe below, and you can see the actual code at Github.
Each frame, WideEyes loops through VBA's RAM and checks to see how many bytes have changed in value since the last frame. It returns the number of frames back to the brain. WideEyes treats this as an "interest value", representing how much fun it's having with the current game.
If a certain number of frames have elapsed (60), WideEyes creates a savestate and stores it in a list.
If a certain number of frames have elapsed (30), WideEyes checks to see if those frames have passed a certain threshold of interest. If they have, awesome. If they haven't WideEyes resets the game to the last savestate and then pops it out of the list of saved savestates. This lets WideEyes keep on backing up if it really gets stuck or bored.
WideEyes can now choose its input. WideEyes chooses its input based on what worked well in the past (it has a very memory of 200 frames).
WideEyes loops through its memory of past inputs and results, and grabs the top 9 values. WideEyes purposely uses a broken algorithm for picking the top 9 values that occasionally gives it bad results. This increases the natural entropy of those results (more on this later).
If the combined "interest values" of the top 9 frames are not above WideEyes' threshold of interest, WideEyes just presses random keys.
Otherwise, WideEyes loops through the inputs returned from the previous loop and builds a key combo to press based on a scoring system that pits certain keys against each other. For example, up and down can't both be pressed at the same time, so whichever one appears most often in the 9 best frames is the key that's pressed.
Finally, WideEyes holds down those keys for an arbitrary amount of time (in case they have a delayed effect). I used 3 frames for this, but it could be just about any number. It seems to vary from game to game.
I was pretty happy with WideEyes at the time. It could do some interesting things with Mario. It produced slightly better than brownian motion (which was far bettter than anything I could have hoped for when I started the project), and it had enough (percieved) personality traits that it could demo nicely in small 5 minute presentations.
This is a stupid algorithm, and it shouldn't work as well as it does. There are a couple of reasons why it doesn't conform to expectations.
This approach allows WideEyes to rapidly change its strategy whenever necessary. If WideEyes runs into a wall, it almost instantly changes its strategy.
There's a constant high variance in what buttons WideEyes presses. I mention this a couple of times above, but a few of WideEyes algorithms are purposefully broken in ways that increase entropy over time. This introduces a natural decay into WideEye's memory that works pretty well to keep information changing in interesting ways.
Because of this WideEyes is almost constantly forming patterns of some sort, which fits that random data into something that looks more deliberate than it is.
WideEyes keeps track of its progress and doesn't reset back to the beginning of a level when it dies (it can savestate anywhere and correct its mistakes).
All of this works together to make WideEyes very good at game genres like Mario, specifically because Mario is using a lot of repetitive motion, and usually the way to make progress can be summed up with very simple formulas that change quickly. You can make a lot of headway by just holding down right and jumping randomly (although you'll die a lot).
Similarly, in Mario, these strategies are usually based on very recent actions - you die because you ran into something a couple frames ago, not because you made a poor move 30 seconds ago. Different games give WideEyes more trouble (it could never play something like Pokemon), but I was able to essentially throw only games at WideEyes that I knew it could handle.
When Piglet was born and I started to redefine how WideEyes worked, I threw out a lot of this math, and a lot of existing performance, because these systems are only deceptively good. In order for Piglet to make serious progress on a game and become truly competent she'd need to drastically redefine how she looked at games.
In this series of articles, I discuss the origins of Piglet, how she's worked at various points in her development, and what some of the design philosophies that went into her development were. If you're interested in learning more about the current version Piglet, check out her site at http://danshumway.github.io/Piglet/. Piglet is currently being rewritten to run in a NodeJS environment and will be relaunching soon!
Imagine you're blindfolded, and handed a controller, and told to play a game. Every time you press a button something happens in the game, but you can't see that happening. Your only interface with the game is that you have a friend sitting next to you - and whenever something on the screen changes, he shouts in your ear, "Hey, something changed!"
It works surprisingly well.
Piglet is a machine-learning based AI that tries to play classic gameboy and gameboy color games using a variety of techniques. She's built using Lua, plays her games using the Visual Boy Advance platform, and for the past month I've slowly starting to stream everything she does on Twitch.tv. Oh, and she's open source, so you can see all of her code and guts on Github.
I originally started work on Piglet under a different name, WideEyes, at a hackathon back in early February of 2014. At the time, I was just starting to dig deeper into a couple of different scripting languages, and I was looking for an excuse to start actually working with APIs and other applications. It was really bothering me that I was digging very deep into how specific languages and data structures worked, but not how to actually leverage them to do any cool things with other existing applications or programs or services.
I had also been following a couple of different projects that were ridiculously fascinating to me, including Playfun (see below), and Twitch Plays Pokemon. In particular, I was really intrigued by the way the Tom created a personality around Playfun and humanized its reactions.
I wanted a piece of that cake. Well. I wanted a different cake that shared similar qualities, but branched out in significant ways, like it had chocolate frosting instead of vanilla, and it was a different shape because I used a different pan to bake it.
Playfun had some weaknesses. It wasn't real time, and it was primarily concerned with finding goals, not learning the rules of the game itself. If I was was creating an AI, I would want my agent to form an idea of how to play optimally, not just what optimal play looked like.
What those concerns look like in a practical sense when building an AI are listed below.
Coming up with goals shouldn't be the focus or most complicated part of the program. Accomplishing those goals reliably is.
Piglet should be able to run in real time (as opposed to executing a pre-determined strategy, similarly to something like Playfun.)
Piglet should fulfill a particular telos when she runs of experimentation and "proper" play - ie. Piglet should have underlying philosophies of how she plays games. This one is a bit more tricky, but it does have a lot of application to how Piglet's code is laid out and what algorithms I choose.
The AI I decided to build was named WideEyes, and the way I decided to approach building it was by modeling how (I thought) humans played games, at least in a very simplistic manner. WideEye's goal would be to avoid boredom.
Visual Boy Advance will return around 65,000 individual bytes - not actually very much information. I had never worked with reading RAM before, so I knew my algorithm needed to be pretty simple. My goal was to get Piglet moving on the screen in an interesting way, everything else was secondary to me getting started with Lua, learning how to interact with Visual Boy Advance, and so on.
The basic program loop I decided on is describe below, and you can see the actual code at Github.
Each frame, WideEyes loops through VBA's RAM and checks to see how many bytes have changed in value since the last frame. It returns the number of frames back to the brain. WideEyes treats this as an "interest value", representing how much fun it's having with the current game.
If a certain number of frames have elapsed (60), WideEyes creates a savestate and stores it in a list.
If a certain number of frames have elapsed (30), WideEyes checks to see if those frames have passed a certain threshold of interest. If they have, awesome. If they haven't WideEyes resets the game to the last savestate and then pops it out of the list of saved savestates. This lets WideEyes keep on backing up if it really gets stuck or bored.
WideEyes can now choose its input. WideEyes chooses its input based on what worked well in the past (it has a very memory of 200 frames).
WideEyes loops through its memory of past inputs and results, and grabs the top 9 values. WideEyes purposely uses a broken algorithm for picking the top 9 values that occasionally gives it bad results. This increases the natural entropy of those results (more on this later).
If the combined "interest values" of the top 9 frames are not above WideEyes' threshold of interest, WideEyes just presses random keys.
Otherwise, WideEyes loops through the inputs returned from the previous loop and builds a key combo to press based on a scoring system that pits certain keys against each other. For example, up and down can't both be pressed at the same time, so whichever one appears most often in the 9 best frames is the key that's pressed.
Finally, WideEyes holds down those keys for an arbitrary amount of time (in case they have a delayed effect). I used 3 frames for this, but it could be just about any number. It seems to vary from game to game.
I was pretty happy with WideEyes at the time. It could do some interesting things with Mario. It produced slightly better than brownian motion (which was far bettter than anything I could have hoped for when I started the project), and it had enough (percieved) personality traits that it could demo nicely in small 5 minute presentations.
This is a stupid algorithm, and it shouldn't work as well as it does. There are a couple of reasons why it doesn't conform to expectations.
This approach allows WideEyes to rapidly change its strategy whenever necessary. If WideEyes runs into a wall, it almost instantly changes its strategy.
There's a constant high variance in what buttons WideEyes presses. I mention this a couple of times above, but a few of WideEyes algorithms are purposefully broken in ways that increase entropy over time. This introduces a natural decay into WideEye's memory that works pretty well to keep information changing in interesting ways.
Because of this WideEyes is almost constantly forming patterns of some sort, which fits that random data into something that looks more deliberate than it is.
WideEyes keeps track of its progress and doesn't reset back to the beginning of a level when it dies (it can savestate anywhere and correct its mistakes).
All of this works together to make WideEyes very good at game genres like Mario, specifically because Mario is using a lot of repetitive motion, and usually the way to make progress can be summed up with very simple formulas that change quickly. You can make a lot of headway by just holding down right and jumping randomly (although you'll die a lot).
Similarly, in Mario, these strategies are usually based on very recent actions - you die because you ran into something a couple frames ago, not because you made a poor move 30 seconds ago. Different games give WideEyes more trouble (it could never play something like Pokemon), but I was able to essentially throw only games at WideEyes that I knew it could handle.
When Piglet was born and I started to redefine how WideEyes worked, I threw out a lot of this math, and a lot of existing performance, because these systems are only deceptively good. In order for Piglet to make more significant progress she would need to drastically redefine how she looked at games.
ript>
Read more about:
Featured BlogsYou May Also Like
.png?width=300&auto=webp&quality=80&disable=upscale)
