In-House Engine Development: Technical Tips

During the work on our in-house game engine, the ABYSS Engine, I have come across a few rules, ideas and tips that help us make sure we have a nice work environment, clean code base and neat project setup as well as an overall high-quality software product.

Most of these rules will probably not only apply to games, but to all sorts of software. At Deep Silver FISHLABS, we apply them mainly to our engine and tools written in C++. This is what I have most experience with and I am therefore going to focus on these types of projects.

1. Separate Projects

Keep the engine in separate projects from your game. They should be independently buildable and the game should be linked to the engine. The game is a layer on top of the engine. As such, it may access the engine, but the other way around - the engine accessing the game - must be impossible. If it is not, someone will eventually introduce such a dependency.

Going further, split up the engine into independent layers that are each usable on their own. These layers are, again, separate projects and libraries that are linked together. Lower layers cannot access layers higher up in the hierarchy.

This way, you automatically get reusable code and a clear separation of responsibilities for each layer. Also, you cannot end up with temporary hacks like "I will just quickly let the input file stream call into the resource system to let it know a read failed" as such hacks tend to become permanent.

This approach is also called a multilayered architecture, but it is very important to actually have these layers as individual projects and libraries. If you want, you can even distribute the libraries as binaries and save the game developers some compilation time.

2. Separate Public and Private Files

With a software project written in C++, you automatically have public and private files. Your .cpp source files are the private files that no one else needs to see (users only need it in a compiled state) and your .h/.hpp header files are, by default, the public files that users need to actually use the engine. These header files are your public interface.

As this separation already exists in a way, you should also make the distinction what is public and what is private exist "physically" in the filesystem. Move your header files to a folder called "include" and your source files to a "source" folder. For any other project linking your library, only point it to the include folder. That way, the users will never even be able to see your private source files.

The next step is to move all header files that are not used publicly to the source folder as well. Any class the user does not need to know the internals of, which can thus be used via a forward-declaration only, should be moved to the source folder. Of course, you keep all classes that do not implement an interface, all template definitions, and all functions that should be inlineable in your public header files.

In some cases, you can even move whole class definitions to the source folder. This is often possible if the class implements an interface. In such cases, you just need to provide the declaration of a function that allows to acquire an instance of that specific interface implementation. The implementation of that function can be found alongside the class definition in the source file.

The great advantage of this approach is a clear and concise interface for your engine. No superfluous information or data is exposed. Therefore, your code becomes easier to use. Coupling is also reduced as the user cannot depend on implementation details, which makes it easier to change those details later (if necessary). As a side effect, you get smaller and fewer includes. This leads to faster compilation.

3. Maintain Build Configurations for Specific Purposes

We all know the "debug" and "release" configurations that normally come with template projects created by various IDEs. For serious projects, these are often not enough as there are more scenarios for your game/engine.

Normally, you would use the "debug" configuration during development, but with games this is often not practical because performance is lacking. Instead, you start using the "release" configuration while implementing new features, but then the game crashes and you cannot debug. So, you enable debug symbols for the "release" build. But now you need a new configuration for actually releasing the game, because you do not want to ship your debug symbols. Also, for QA you need a build with good performance, but also more error checking than you would like to ship to the end user.

To sort through this build mess, you need at least three distinct build configurations:

To be clear, debugging information is used to map compiled code back to C++ symbols as well as files and line numbers. Even if you do not build this information into shipping builds, you should still keep it around separately from the binary. If you do, you can later re-symbolicate any stack traces you receive from players of the game. Debugging functionality includes things like asserts or extra bounds checking.

You can also get more exotic and define these additional configurations:

I am sure you can come up with a lot more by tailoring the configurations towards the specific workflows in your studio. The advantage here is that you have the optimal build for any development task like testing, debugging, profiling, adding functionality and so on.

4. Generate Your Projects

As game/engine developers, we often need to support a wide array of different platforms and project files of any IDE are notoriously difficult to merge. Resolving conflicts in them is a nightmare. In order to cut down on the complexity of maintaining these project files for all platforms and different build configurations, it is a good idea to generate these project files from one file describing what they should look like.

There are a lot of different tools out there that allow you to do this. There is CMake, Premake, Generate Your Projects, Waf, and a lot more. We currently use CMake, but used Premake before.

The idea is to describe what files your projects comprise, what build settings they need and what dependencies your code has. Then you run the tool and out comes a project for your favorite IDE that you can just open and build like you would normally do.

The project description is human-readable and is comparably easy to merge and fix by hand if something goes wrong. Of course, it is also possible to specify such things as platform specific compilation flags, which sometimes becomes necessary. 

Generating your projects from simple descriptions makes it easier to have multiple projects and link them together to form the final executable.

Summary

This all comes down to having a clear structure and using the right tools for the job. Always strive to make your project easier to understand, easier to use for a given purpose, and easier to modify.