Multiplayer Is Hard

This is a post duplicated from the Digitanks website.

Thanks to everybody who tried out the demo I posted this week. I’ve already started to receive lots of feedback on it and suggestions on how to improve it. Your input is hugely valuable to me. Some of the most common issues were the lack of good camera control and the lack of a good variety of tanks, issues which I’ll have resolved by the next demo.

Part of the original plan for Digitanks was to forgo multiplayer entirely. Multiplayer games are much more difficult to develop and test by an order of magnitude. There are a number of problems that need to be dealt with to develop multiplayer games. For one, data that is created, modified, or destroyed on the server needs to be sent to each client. Then, the actions of every client need to be sent to every other client. It also adds a layer of complexity to every feature in the game. In addition to the art, programming, usability and testing that every feature needs, these all need to work just as well in a multiplayer environment as a singleplayer one, which requires much more additional work, and commonly results in a headache and a hernia. Plus, I really just don’t enjoy writing multiplayer code so much. So, I decided to just leave multiplayer out of the equation to get Digitanks done faster.

Of course that plan had to change. As I started playtesting the game, I asked each playtester, “How important to your purchase of the game is an online multiplayer feature?” and the number of respondents who stated that they wouldn’t purchase the game without online multiplayer stands currently at 88%. (n = 13) This is isn’t true for all games, for example I don’t think many people would require World of Goo to have multiplayer, but it seems that it’s very important for the kind of people who enjoy Digitanks. The recent Trine postmortem mentioned that they think they would have sold something like three or four times more copies of the game had it included a multiplayer feature.

Well, that’s a tough call for me, since multiplayer can be such a difficult beast, how can I justify the extra amount of work that’s required? I built the engine myself because I assumed that I wouldn’t need multiplayer, but now I’m faced with an interesting choice. I have to either accept reduced sales without multiplayer, or figure out a cheap and effective way to add multiplayer into an existing game. I have the advantage of the fact that a turn-based game is not as difficult as most to write multiplayer for, since there aren’t really many realtime events that need low-latency or immediate handling, so I likely won’t have to deal with multiple threads or synchronizing entity data across multiple clients if I can just inform each client of the other’s decisions.

The first step is to look at a number of multiplayer libraries to see which one is the most appropriate for me. To Google! The first result for a search on “game networking library” is a tool called RakNet. It looks like a well-built, professional game networking library. It’s used in a lot of major game studios according to the website, and is free for indie game developers. I like the looks of this tool, but the license agreement gives me some pause. For the record, there are two types of license agreements that scare me: proprietary license agreements and GPL. LGPL is annoying since it requires dynamic linking, but I use one or two LGPL libraries with no problem (DevIL and SDL if memory serves) but I can deal with it. The problem with proprietary license agreements are that it’s tough to tell what’s really in them without a lawyer, and they can change at any time since they’re mostly designed to protect the authors. So, we’ll hold RakNet on the back burner and see if there’s anything else we can use.

GNE is the next option. It’s LGPL, putting it in much the same category as RakNet. The problem I see with both RakNet and GNE is that they’re a bit too high-level for me. I don’t want a tremendous amount of overhead with whatever tool I use, and both of these tools seem like they have a very high-level object-heavy way of doing things that I don’t necessarily agree with. Although I love object-oriented development, I tend to shy away from libraries that expose an object-oriented interface, a simple well organized series of c-exported functions is more desirable. I’d prefer something simple that I can plug in and get working quickly. The thinner my networking level is, the easier it will be to resolve problems I encounter with it. So I keep looking.

There are a couple of other libraries in the search result, but they’re all either outdated or just plain crappy. However, scouring some forums and mailing lists reveals a small library that everybody says good things about named ENet. It didn’t turn up on the search for game networking libraries, since nowhere on its front page does it even mention that it’s intended for games. Whereas the other libraries handle object data replication and things like automatic updates (stuff I don’t really need) all ENet does is provide a simple and lightweight wrapper over the operating system’s networking. That works out great for me, since using OS-specific calls is out of the questions (we do eventually want to target other platforms!) and the additional features that ENet provides don’t add any significant overhead or programming complexity.

ENet looks good, but there’s a couple gotchas that need sorting out. The first one is the license. Opening up the LICENSE file revealed my favorite kind of license, a fully permissive MIT style license. This style of license basically says, “Do whatever you like with this software, so long as you don’t fault me if something goes wrong.” I LOVE that! Library authors take note: I am much more likely to use your software if you don’t impose arcane rules about how and when it can be copied. The last issue then is the maintenance. If the library is no longer used or actively maintained and has bugs, it’s useless. Looking at the download directly, I see that the most recent version was uploaded one day prior. Fantastic! Then the last question to get answered is, how good is support? Does the author help people who are having trouble with his software? Well as it turns out, there’s a mailing list where the author is an active poster, and an IRC channel on FreeNode where the author is present and commonly answers questions from people like this hopelessly confused author, yours truly. ENet it is, then.

So I have my library all picked out. All I need to do now is figure out how I’m going to structure my code most efficiently. The key here is time, how can I get this done fast? My players demand a multiplayer experience and I want to give it to them, butI want to do it in a way that will cost me the least amount of development time. I basically have two options.

Choices… I’m always a fan of spending a little bit of time up front in the planning and tools development stage in order to speed up the actual development work. It’s a good plan and the time spent laying foundations and doing ground work usually pays off in the end with reduced development time, but that only applies to tasks that will be repeated multiple times. An IBM engineer once told me that if you’re going to do it once, just do it, but if you’re going to do it more than once, automate it. The sentiment is correct, but I don’t know if it’s really a time-saver to automate something if you’re going to do it twice. To me, you automate something if you’re going to do it 200 times, but I’m not sure Digitanks is that kind of project. Once I get the basic tank movement and interaction networking code in place, there’s not really much more that will need to be networked. Digitanks will be released by the end of the year and I’m likely the only person who will be programming on it between now and then, so there’s no need to automate everything.

The plan is to use each of the important functions in the game as proxies for network activity. For example, any time the FireTank() function is called, the networking layer will replicate that message to each client connected, so that same function is called with the same parameters on every client, and every client will fire their tanks. The system relies heavily on making sure that the same functions are called on all clients to retain the same state of data across all instances of the program. This design means that a simple programming mistake can make it easy for one or more clients to get desynchronized from the rest, so I’ll have to be vigilant in making sure that I don’t leave anything out. It’s been pretty tough to put all of that in place, but the results are actually pretty promising. I’m five days into it and already I have this to show for it:

multiplayer

What you see here is the same scene being rendered from two different angles in two different clients. Some of the tanks have been moved around and they all have aiming targets and different values for their power usage, and as you can see all of that data is consistent between the two clients. Multiplayer can be tough, but with a simple enough system, I’ve lowered the complexity to a manageable level. End result: You can expect Digitanks to feature online multiplayer when it’s released in September. I hope it sells a lot of copies!