Q&A: Dissecting the development of Dwarf Fortress with creator Tarn Adams

One of the best uses of a high-powered processor out there for entertainment purposes has long been Dwarf Fortress, the game that makes an entire world out of ASCII characters, and will happily consume a gigabyte of your RAM and a good chunk of your processor cycles to bring it to life.

But unlike some other games, DF feels like it needs everything it requires. Its exhaustive calculations create an entire world, with buildings, towns, merchants, rivers, volcanoes, monsters and, of course, dwarves. If a single person created all these things it’d be an amazing achievement; Dwarf Fortress is a program that creates all those things on its own.

Creator Tarn Adams has agreed to answer some of our questions about the creation of what is still, despite the presence of a number of imitators, an entirely unique game.

Gamasutra: I hear that Dwarf Fortress is coming to Steam and itch.io in a paid version with an easier interface. I heard the reason was concern over possible future medical costs. Do you find it challenging staying afloat as an indie developer?

Adams: Yeah, a lot of work goes into sticking around, and almost everybody ends up needing to think of creative ways to continue to pay the rent and other basic expenses.  And something could always come up that is just too much to handle.  We've been lucky so far, but we've also had to make some large shifts over the years, whether that's changing jobs or moving to Patreon, or to Steam and itch now, or making crayon drawings, which is somewhat far removed from our video games.

What does your day-to-day coding on Dwarf Fortress look like? What languages do you use? What libraries? What IDE, if any? What does your dev machine look like?

I just work with a Windows 10 Toshiba laptop that isn't notable.  I use some horrifying amalgam of C/C++ in MSVC Community.  I'm using OpenGL by itself for the legacy version, and SDL for the main one, with FMod for the sound.  I don't use anything else on Windows, just some of the headers that comes with MSVC (and before) that I've been using for decades.  I'm not sure what's going on over in the Linux/Mac versions, since I don't develop regularly there.

Dwarf Fortress has been in development for almost 17 years now, and its codebase has got to be gigantic by this point. On my machine, making a world just on default settings takes well over 1.2 gigabytes of RAM. One problem with mega-projects like DF is when the project gets too large to completely fit in your brain at once.  What strategies do you use to keep it comprehensible and understandable enough to be worked on?

I have a consistent way of naming things, and I don't skimp on longer variable and function names so that everything is readable, even after years away.  Generally, I'm mindful of being kinder to my future self.  My comments are all made toward that purpose.  "Find In Files" sees a lot of use.  But there are times when I do have to re-familiarize myself with what's going on, when an old system needs to be extended or a bug needs to be fixed, and it can take an hour or more sometimes just to collect the pieces.  That allows me to leave additional useful comments that I might not have considered originally.

 I remember that Threetoe (Tarn's partner in developing the game) writes stories that you then try to create a game engine in which they could occur. I still find this an inspiring way to work. Are there any stories that you've had to reject as too difficult? Have you ever found the plot of one of them duplicated exactly within the game?

Ha, I think they are all too difficult in some sense.  Character motivations and goal-setting and etc. continue to lag far behind what they do in stories.  It's still a useful process though, since there are some easier story-generating elements that stick out, and we can also orbit the core character mechanics even if we never attain them.

Wikipedia reports that the game's version number (currently .44) indicates about how far along you are in completing the game (that is, 44%). What is up next for Dwarf Fortress? Do you feel any anticipation over what's to come? What major features do you have left to implement?

I'm finishing the villains release up over the next some months.  That should be pretty entertaining.  Then we're implementing the graphics and QoL stuff for the Steam/itch versions.  After that, we have some siege improvements and other work before entering the Big Wait.  The latter'll be the largest DF restructuring and addition ever, I'm pretty sure, allowing us to generate creation myths and have entire procedural magic systems, as well as having multiple view windows open in different parts of the world, etc.  That'll be great.  Then there's the property/law/customs/etc. release.  After that, the order isn't set, but it'll involve the economy, boats, and other major missing components.  There's a lot left to do!  We aren't even halfway to 1.0 yet.  And 1.0 isn't really completing the game...  we just might not have a lot of time left by the time we get there.

In gaming, there's the balance between Narrative and Simulation, between the pre-written story most games have, and making a deep world with a set of rules that can make many stories possible. Dwarf Fortress is one of the greatest arguments towards Simulation, I would say. Do your characters, either during the game or in world gen, ever do things that you wouldn't expect? Do you have any interesting/memorable examples to give us?

All the time!  In part, just because it's hard to keep all the rules in your head when you are just messing around.  All of my memorable stories are bugs though, since I hardly get a chance to play the game for long now, so they are in one sense fundamentally uninteresting from an emergence standpoint (even if they are amusing.)  The forums and streamers and so forth have the good stories.

The line between micro and macro: why did you draw the line where you did, between what dwarves do for themselves and what you direct the fortress to do?

It's a difficult balance and not always straightforward or consistent, but the current conception is that we want the player to be the "official will of the fortress" while the dwarves also exercise the autonomy they should be expected to have outside of their official duties.  This allows them to be actors in their own stories, which is a large part of where emergent narrative comes from.  At the same time, the player should be able to direct the general flow of their part of the game (this isn't crucial, really, but it's thought to be more fun than just watching a simulation, oftentimes, ha ha.)

These two goals can run into each other, and often it relates to the player continuing to have fun with the game - for instance, if an emergency lever really needs to be pulled, the job priority system can practically force a dwarf to go and do it, autonomous or not, whether it feels like they should really 'know' to do it or not.  We've had problems in the past adding too much bureaucracy, when we had a quartermaster-type dwarf for instance handing out equipment.  It was just too slow and fiddly and prone to confusion and bugginess.  Thinking about what your individual game mechanics add to potential stories is important, and the quartermaster pretty much didn't have a role there.

What steps does the program go through in building a world?

It allocates the memory for the map.  Then it chooses what sort of pole (e.g. north, south) it is going to have (or respects the parameters fed by the player, throughout.)  The basic map field values (elevation, rainfall, temperature, drainage, volcanism, wildness) are seeded along a grid of variable size, respecting various settings (oceans, island sizes, other variances, etc.), and then filled in fractally.  The poles vary the temperature, and it selects some points for the highest peaks.  Here it does a first pass to see how it is doing, and attempts to adjust some altitudes to fit the map within the desired parameters if it missed.  The world can be rejected at this point if it is unfixable, and it tries again.

The first derived field, vegetation, is then set based on elevation, rainfall, temperature, etc., and it tests for biome rejections if the map's biomes don't satisfy the ranges set in the parameters.  The mid-level elevations are smoothed at this point to make more plains areas, and volcanoes are placed respecting the hot spots in the volcanism field.

Then we enter the erosion and river stage.  Small oceans are dried out, and it locates edges of mountain sides where it can run test rivers.  It also places the camera on one of these so the player can watch the process.  Many fake rivers flow downward from these points, carving channels in the elevation field if they can't find a path to the sea.  Extreme elevations differences are often smoothed here so that everything isn't canyons.  Ideally we'd use mineral types for that, but we don't yet.  Lakes are grown out at several points along the rivers.

Elevations are smoothed again from the mountains down to the sea, and the peaks and volcanoes do some local adjustments.  Now that the elevations are finalized, it makes adjustments to rainfall based on rain shadows and orographic precipitation.  Temperatures are reset based on elevation and rainfall and the dampening effects of forests, and it uses the new values to set the vegetation level one final time.  Salinity values are set for the ocean and tiles neighboring the ocean.

Now that everything has settled down, we can detect the limits of the final biome regions and give them names and their own identity.  We also add the geological layers and the underground layers here, though the geological stuff should really be earlier, as previously mentioned.  There's a final verification process against the parameters here, to make sure it hasn't drifted too far afield from what the player wanted.  Once that's done, it generates the initial wildlife populations in each region, and sets some weather variables.

History itself can begin at this point.  Civilizations and caves are placed.  It's a bit complicated to go into what happens after that, but the basic idea is that there's a giant zero-player strategy game going on with somewhat loose turn rules and bad AI (but thousands of agents), and history is just a record of that.  Procedurally generating stories by recording a log of a simulation is a valid enough approach, though it has drawbacks, of course.  It's a lot of work, you need to do post-processing or investigation to find any good moments you'd like to highlight, and if you don't have enough dynamics and mechanisms, the output can be boring (in any number of ways.)

One of my favorite facts about DF is how it doesn't have an explicit hit point system at all, that everything about a character's strength and damage is part of a complex body part model. What advantages would you say such a system offers? If one did decide to create one, what obstacles would you warn them about?

It provides more specific relatable story moments, lasting consequences, and provides more consistency throughout the systems.  It's possible to overdo it, and we have in some ways, at least for now, while some of the material properties aren't used elsewhere.  There are also ways to soften the system, which you can see in games that, for instance, have a stamina/energy/hp pool, but specific wounds occur as either criticals or the consequences of reaching zero or near zero in the pool.  That can all work, depending on the game.  For us, being as anti-numeric as possible is ideal, since numbers usually make for poor stories.

What does the main game loop look like?

In dwarf mode, say.  It starts with some announcement checks and considers autosaves, etc.  A lot of the rest doesn't happen every tick.  Every hundred ticks, for instance, it'll check job assignments and 'strange moods.'  Armies are moved on the world map.  Every hundred ticks (staggered from the job check), it handles job applications by dwarves, a kind of invisible auction that it uses to manage the various competing priorities.  Every ten ticks it advances the seasons, which actually does all kinds of things with the weather and map (both locally and in the world), and it also does a check for certain plot element advances (diplomats, sieges, etc.), and it checks whether your fort is still alive.

Then it hits some things that it does every tick.  Fluids and other map tile information is advanced (though there are various optimizations here so that not every tile is necessarily checked every turn, and there are various flags so that entire sections of the map can be skipped if nothing has happened.)  Vermin running around are updated.  Other 'events' on the map, like active fires, are handled.

If a flag is set, wounded/thirsty/hungry dwarves that can't care for themselves get an update, and every so often, dead dwarves 'think' about their burial arrangements so that jobs can be set.  Caged and chained creatures update their thoughts and situation periodically.

Then creatures leave the map if they are set to at the edges.

Every fifty ticks, staggered from other updates, all of the taverns, temples, libraries, etc. get their information updated.  Stockpiles, staggered on a different tick, also work this way.  Similarly with storage job creation, though that process is complicated with various optimizations and lengthening ping times and so forth, and it's still slow, since at some point, 50000+ boulders will cause trouble.

Every thousand ticks, objects that have been marked for deletion and removed from the game are actually deleted and freed.  This happens more often with items, every fifty ticks, along with a building use check (mostly updates for wells and some other flags that often need checking.)

We hit another every tick update at this point.  Projectiles are moved.  Activities (which range from dances to martial training to storytelling) get updated as needed.  Dwarves and other creatures decide on and advance their immediate actions (movement to an adjacent tile, working at a workshop, etc.) -- the bulk of their AI (outside of job selection) is here.

Every hundred ticks, items are rotted.  Every tick, vegetation is advanced (though there are various staggers and flags here.)  Building states are updated every tick as needed, and minecarts are moved.  Hauling routes are advanced.  Temperature is updated (there are various optimization flags here, but this is still an unfortunately slow process.)

Finally the camera is updated to follow the creature you are following, if any.

Dwarf Fortress uses a grid-based tilemap to depict its world, a simple, efficient and effective means of representation. I notice that there's many ways a tile could be drawn, based on whether it's a creature, what the creature is doing or feeling or if it's dead, whether there's more than one thing in a tile, whether there's something above it, whether it's flowing water, or if it's entombed by rock. When DF gets down to deciding what to display in a tile, what does it do? How did you optimize it?

It's just a character (byte) with another few bytes of color, so it's not expensive to just change your mind and replace a decision before the final print, rather than trying to decide all at once, and most tiles just want a single ground/wall character anyway.  It just goes bottom to top, 'elevation'-wise in some sense (that is creatures above items, items above the ground), replacing the decision at times.  Still, there are various flags and helper arrays and so forth, though more can be done.  There is a little array of possible units to be printed at each tile, so that it can do a flipping animation that allows you to see every unit (over time) even when the game is paused.