Das Tal’s Backend & Exit Games’ Photon

This article gives an overview of Das Tal's infrastructure and some details about the game server. Be aware that this is just our way of building an online game with our very small team. If you have an opinion or comment about any of this I would be happy to hear from you. This article was first published on our dev blog.

A little background on the game design

Our game is designed so that we have lots of different small, isolated game worlds. We are talking about 1k - 2k registered players with 100-200 CCU (concurrently connected players) on one world. The reasons for the size are that we want to time-box worlds, have intimate player communities (within one world) and that we want to keep the technical complexity as small as possible.

The core gameplay is focused around a very fast and twitchy gameplay.
So we are somewhere in the middle between a bunch of Counterstrike servers and  a “classical” WoW-like MMORPG structure with a huge world made of connected instances. Here is a gameplay example:

A birds-eye view on the infrastructure

This is an overview of the services that are somehow related to either the live operation of the game or its the development. Most of the services are already up and running although some only exist as dummies.

LIVE

DEV

A lot of these services are quite straightforward. Most MMO-like games have them - although probably in different tastes.

The game server

All these parts are important but the heart of all this is the game server. After evaluating different things we decided to use Photon server.

Mainly because we can easily share code between server and client (C#), because it is fast and well established. During pre-production I wrote a little bit about the game server middleware evaluation we did. 

Overview

Our game server is a fully authorative server. The core game loop (simulation) runs at about 15 ticks/second. Because multi threading is incredibly hard we decided to limit the core game loop to just one thread. Designing and maintaining a single threaded game loop is much easier. There will, for example, be fewer strange timing bugs during high loads (like duplicated items).

The downside of this is that our core game loop can not scale horizontally and we will finally hit a point where we cannot scale-up one game world anymore. But our game worlds are small by design so that’s OK.

Other things like network IO, calls to external services, long-running calculations (pathfinding) or saving the world data run in different threads. Actually all these things run as a fiber using Photon’s internal fiber implementation. 

We use a 2D grid index for client interest management. This is necessary to reduce traffic and CPU load to a manageable level. 

Our gameplay is based on the entity system framework Artemis and it uses Farseer for collision and movement.

For small messages between client and server we only use photon’s own serialization (key value list of basic data types). Bigger and more structured/complex data gets serialized by protobuf and encoded as a byte array in the photon protocol.

The game state completely fits into memory. Therefore we just serialize it completly using protobuf and save it into a file. This is only feasible because each game world is small and limited in time and will not exponentially grow.

My tip: Next to the protobuf-encoded file we dump the complete game state as one big YAML file. This way we can easily inspect the game state with just a text editor. No extra tool needed.

All logging is done via log4net which is contained in the photon server. Windsor is our IoC container. 

Our unit test setup uses the Visual Studio Unit Testing Framework and is based on the setup in the photon MMO demo example.

Peer & service interaction

A peer represents a connected client and acts as a gateway to communicate with the player. Each peer has a fibre that processes the incoming and outgoing messages. Basically all peers exist in isolation. Communication between different peers is done using services. For example after finishing the authorization the peer registers at the chat service and can send and will receive chat messages in the future.

A peer has a protocol state object that represents its current state in the protocol. So if a client connects and joins the game it goes through these steps (from the view of the client):

Game service

The game service runs the fiber for the core game loop. It is the gateway between ingame (things running on the game fibre) and not ingame. So if a peer wants to enter the game it calls “gameService.Enter(peer)”. Character movement messages go through “gameService.OperationBodyMovement(peer, movement)”. All these commands get queued up in the game loop fiber and get executed between ticks (calculating one step of the game simulation).

Another job of the game service is to snapshot the current game state (synchronous) and hand it over to the map storage service which serializes and stores the state (asynchronous).

Core gameplay loop

This snipped shows how the tick method gets called within the game service.

The heart of the game loop is the Artemis entity system. The entity system and all code that interacts with it is shared between client and server (the system that handles the game network synchronization is an exception). This is so the client can use the same data structures and methods if necessary. Currently the client only uses data structures and some check methods from the shared gameplay code. Artemis contains systems for the different gameplay parts (eg. ASCharacterRegs for health and energy regeneration, ASCharacterCast for casting skills).

A very important system is ASFarseer. It calculates collision and movement with the help of the Farseer library.

The ASSpatialIndex system contains different spatial indexes based on grids and is necessary for efficient client interest management. There are also special grid indexes to handle the footsteps (eg. protect regions against “footstep flooding”).

The ASNetwork (called “game network outgoing” in the diagram) handles all the game state synchronization with the clients.  This component has access to all the peer instances that are currently active within the game service.

The following steps show the interaction of these components on the basis of an example:

Example: A player moves her character and sees the result on the screen.

That is all for this week. If you have any questions about how we implemented things or if you have feedback then please get in touch with me at [email protected].

- sebi