Full disclosure: I am working with Valve under contract on an ongoing technical writing project, including, but not limited to, this blog post. I am neither an official employee nor a representative/spokesperson for Valve, and any opinions or personal views expressed herein are my own.

Back at Steam Dev Days 2016, Valve graciously invited me to share the stage in explaining the Steam Controller to developers:

That talk was pretty general and big-picture, so I would like to follow it up with some more practical tips derived from my own experience adding native Steam Controller support to my own game, Defender's Quest: Valley of the Forgotten, and explain some of the many benefits you can get from doing the same.

One hesitation some people might have for adding native Steam Controller support is -- why go to all that work just to support one device, and a kind of weird one at that? Isn't it enough to just support XInput and be done with it? And what if the game is mouse/keyboard centric? Won't it be a pain to adapt it for controllers?

Today I seek to to answer all those questions and more.

The Input Revolution

The biggest news is that the Steam Controller initiative isn't just about one physical device any more.

When you hear the word, "Steam Controller", this probably comes to mind:

But did you know all of these are now "Steam Controllers"?

As of January 19th, the latest version of the Steam Client enables full Steam Controller Configurator support for not only the Valve Steam Controller, but also the Sony Dualshock 4, as well as all XInput devices, such as the XBox 360, XBox One, and XBox One Elite controllers, not to mention generic models manufactured by Logitech, et al.

Furthermore, there is already (experimental) support for rigging up DInput devices -- just look, I even managed to get this third-party Wiimote driver working* with the software!

*Just a proof of concept, your mileage may vary.

When you add native support for the Steam Controller API, you automatically get every supported device, a list that already comprises 95% of popular gaming input devices and will continue to grow over time.

And let's not forget these trusty stand-bys, which are also supported by the configurator:

(in the sense that you can assign mouse & keyboard inputs to devices supported by the Steam Controller, though not the other way around at the moment).

The configurator not only lets you use any supported device with just about any Steam game, it also gives players unprecedented control over how they control their games. There's no better example than this heart-warming reddit post from a disabled PC gamer:

But there's a lot to unpack here. What's this "Configurator" I just mentioned? Let's back up for a second and clarify what we're talking about.

Terminology

These are the terms I'll use throughout this document, for clarity's sake:

Redefining Input

It's a gargantuan task to try to add direct support in your game for every single input device out there in a consistent, stable, and flexible way. The closest you can get right now is SDL, which I highly recommend for bringing a strong dose of sanity to conventional input, but the SCC takes things one step further.

Aside: my (personal!) hope is that the Steam Controller initiative will eventually become a new universal -- and open -- standard for PC gaming input. I'm definitely not speaking for Valve on this particular point as I have no idea what their long term plans are.

As I mentioned in my talk, the SCC is built on a few "big ideas:"

The Steam Controller API is built around "actions." Mario jumps in response to a "jump" action from the API; the game has no knowledge of the "A" button. The player controls what means "jump" - it could be "A", "X", flicking a joystick upwards, whatever. The developer can (and should) upload a default configuration for the game, but the player can change it to whatever they want.

But it's more than just customization -- the player also gets features like turbo right out of the box. Even better, as new controllers become supported, more input methods become available without games having to add special support for them. For instance, if Valve were to somehow add support for original Nintendo Famicom controllers, the player could presumably trigger any in-game action by breathing into the microphone.

Here's my point. The Dualshock 4 features a gyro, but a common lament is that most PS4 games don't take advantage of it.

Why not? Probably because they'd have to add specific code to support it, and the XBox One doesn't have the same feature. Plus, design inertia keeps developers from sticking their neck out. This is a serious oversight, because twin analog sticks just don't have enough precision to match the fine-tuned aiming we're used to on PC.

But throw in gyro assist, and you can get pretty dang close. There's no better example of this than the new DOOM game:

Although DOOM doesn't use the native Steam Controller API, the Configurator is powerful enough to do some fun tricks in what's called "legacy mode," a backwards-compatibility fallback for games relying on conventional input.

In this screenshot you can see that each physical input on the DS4 controller is simulating some other device input -- in this case a mixture of conventional gamepad and mouse inputs. The right stick is bound to "Joystick Mouse" which means the game will receive low-level mouse movement inputs from it, and the internal gyro is also bound to mouse movement (The player in the above video is using a similar setup with the VSC, using the touchpad and gyro). The joystick sends coarse mouse motion, and the gyro sends fine mouse motion, and stacked together they create a seamless, high-precision aiming experience you simply can't reproduce with conventional analog sticks alone.

And here's the kicker -- DOOM doesn't have a single line of code to take advantage of this advanced aiming mechanism, it gets it for free thanks to the Steam Controller Configurator. And if you're using an XBox controller that doesn't have a gyro like the DS4 or the VSC? The default configuration just falls back to conventional stick aiming.

So, if the SCC can retrofit this awesome functionality into DOOM just via legacy mode, what's the point of adding official support for the native API? There's several reasons:

To touch on point 3, one of the chief headaches for legacy mode is when a game assumes the player will never need to use mouse/keyboard and gamepad inputs at the same time. If you mix and match these inputs in the SCC and the game doesn't cooperate, you'll see in-game input icons "flicker" back and forth between gamepad and mouse/keyboard hints, if they even work at all.

If your game makes proper use of the native API, you avoid all these headaches and create a much smoother user experience, and the best part is, it's easier than you think.

Case Study

Let's move on to a practical example from my own experience: Defender's Quest.

I made some rookie mistakes early on by underestimating the new system:

The common thread to all these mistakes was one deeply held and entirely false assumption:

So if there's one thing you take away from this article, it should be this:

This isn't to say the Steam Controller is going to best mouse/keyboard players in high-level competitions, but it is to say that if you have a button on the screen, in most cases, the player can just mouse over and just freaking click it. This works especially well with the VSC and the DS4.

You can get the same functionality with XInput devices via the SCC, but since those devices don't have a touchpad or gryo, the player will have to move the mouse around with just the analog sticks. For this case you might consider making sure you have conventional controller-friendly navigation actions available.

All that said, if your experience is anything like mine, you'll find manually supporting conventional controllers via XInput and other standards requires a lot more work than supporting the SCC -- especially if you want it to work cross-platform.

Now, onto the game itself. Defender's Quest makes for an interesting case because:

All of which make it quite challenging to design a good user experience for conventional gamepads.

Let's go through each screen of the game, and see how they look when optimized for Mouse/Keyboard, Gamepad, and the Steam Controller API.

Basic Menu Navigation

In mouse/keyboard land, this is simple -- stick a bunch of buttons everywhere and let the player click on them.

Mouse/Keyboard

Easy enough. I also like to stick tooltips on everything as an extra bit of polish and aid to understanding.

Now onto gamepad input.

Conventional Gamepad

Console RPG's have a well-worn standard for menu navigation: the "finger cursor":

This works best when options are laid out in a grid or list - press up/down/left/right to navigate, "A" to select, "B" to go back. Couldn't be simpler, and it's surprisingly adaptable to a wide variety of menus. We used this in Defender's Quest, and in accordance with The Five Golden Rules of Input as well as my own user testing results, we bound default finger cursor movement to the DPAD as well as the left and right analog sticks.

The other face buttons like X and Y are mapped to commonly used on-screen buttons so you can activate them with a single press:

Steam Controller API

Since most Steam Controller Devices can function just as well as a regular mouse, I could have just exposed an analog action that moves the mouse cursor and a digital action that clicks things, and left things at that. This would have been fine for supporting the VSC and DS4, but it would probably have been a bit awkward for XInput users.

Since I had already gone to the trouble of making a conventional gamepad menu, I went for the best of both worlds. When the Steam Controller API is in use the game uses "gamepad mode", but everything you see on screen is also clickable with the "mouse" whether that's an actual mouse or a Steam Controller Device.

Furthermore, whenever the game senses "mouse" movement, it hides the fixed-position menu finger cursor and instead shows the free-floating finger cursor which indicates mouse-like input, distinguished by a 45-degree tilt:

This lets the player fully control what input style is best for them -- they can navigate the menus with up/down/left/right actions, or using analog actions to move the finger pointer directly and click on the thing they want.

Also, see those button glyphs on the bottom of the screen? When using the SCAPI, they automatically match whatever device you're using. In the previous screenshot I was using a Valve Steam Controller, and so the game receives numerical constants from the API that correspond to VSC button glyph images. Here's what it looks like when using a DS4:

So just add all the right glyphs to your game, hook up a function to poll for an action's button glyphs, and you're done. But isn't Valve adding support for new controllers all the time? What happens if someone uses a new device my game doesn't have button glyphs for?

Got you covered:

Although my game does ship with its own DS4 button glyphs, I temporarily disabled them here to demonstrate a special feature. Here, the game calls the GetDigitalActionOrigins() function, which returns the physical inputs that the player has chosen to bind to that particular digital action (IE, "press X for options").

However, in this example the game receives a value it doesn't recognize because that value wasn't defined when the game was compiled, and thus it doesn't know what image to load. This case is easy to detect in code, at which point the game calls the GetGlyphForActionOrigin() function as a fallback and passes in the unknown enumeration value. This returns a path to a suitable image file for the unknown glyph by grabbing it directly from the Steam Client's install directory:

In other words, it's a forwards-compatible polyfill.

This is a super convenient function whose only downside is that the glyph might not perfectly match your game's art style -- a much better outcome than broken image glyphs, or even worse, a crash! And you can always push an update with new custom glyphs on your own schedule.

Now, let's take a look at the default menu navigation configuration:

First of all, we're in the "Menu Controls" action set. As I said in the talk, all actions are divided into different "action sets", and this is the one that governs basic menu navigation. Action sets let you organize related actions into distinct groups that are only active at certain times so you don't crowd the input space with always-on actions that are only used some of the time.

You'll notice that both the right touch pad and the gyro are bound to the "move finger cursor" action -- I'm using the same "stacked input" method I mentioned earlier with DOOM. If we go into the gyro settings we see this:

This is just a small taste of the little tweaks you can do with the Configurator -- the gyro only activates when the player is physically touching the right pad. Since the right pad is itself bound to the "Move finger cursor" action, this results in a trackpad for mouse movement with seamless and subtle gyro assist, but when the player stops touching it there's no extraneous cursor movement. Most users I tested this on weren't even conscious that the gyro was helping them, but they did notice that the mouse pointing felt smoother and more precise than previous (gyro-less) input configs.

Some users might not like the gyro, or have some specific opinions on how to best use it, in which case they're free to change up the config any way they want, the game will be none the wiser. XInput devices that lack touchpads and gyros won't be able to take full advantage of this feature, but can still be pushed past their original design limitations -- you can still bind the analog sticks to send any kind of input (including mouse input), the results just might not be as smooth and precise as you can get with a VSC/DS4 touchpad and/or gyro.

Overworld

The overworld requires dedicated controls for moving around the map and thus can't simply piggyback off of basic menu navigation.

Mouse/Keyboard

If you want to go to a location on the map, you point and click. If you want to open a menu screen (Party, Options, Tips & Tricks, etc), you point and click. Everything is extremely straightforward.

Now let's adapt this for conventional gamepads.

Conventional Gamepad

Moving around the map works well with gamepad controls - move the character to the location that most closely matches the vector received from analog stick motion. Then press the "A" button to enter the location. Easy!

But wait! User testing revealed some players would always want to use the DPAD to move around the map. This worked in most cases, but it didn't handle certain diagonal edge cases well since the DPAD can only input cardinal directions and it's very rare for, say, RIGHT+DOWN to happen on the exact same input frame. To handle this special case, I added specific input code for this screen that waits a few frames before responding to DPAD input so that it can properly catch diagonal input events.

Okay, a little tricky, but we're done now, right?

Not so fast. That row of handy menu buttons on the bottom are now inaccessible -- the player doesn't have a mouse, so how are they supposed to click on them? I could use a finger cursor menu, but -- shoot, I was already using the left analog stick and the DPAD for map movement.

I was tempted to break the 5 golden rules and reserve one of the sticks for finger cursor menu navigation, but user testing proved this to be confusing. Players naturally expected both the left analog stick and the DPAD to move them around the map, and nobody even thought to reach for the right stick at all (I wound up binding it to the non-essential "scroll map" action).

This left me with no reliable inputs for navigating the bottom menu bar buttons with a finger cursor. Besides, finger cursor navigation would still overload the context of the "A" button -- should it enter the current map location, or click the button the finger cursor is pointing to?

Ultimately I went for the lazy option.

Most of the menu screens are now hidden behind a modal shift. Pressing "X" brings up this popup:

It freezes the map, grabs input focus, and opens a boring, fool-proof finger cursor menu. It's clunky, it adds an extra step, it hides some of the options from immediate view, but it works. And when all the player has is conventional gamepad input, it's the best I could do.

Steam Controller API

Here's how it looks with the Steam Controller API:

It's almost identical to the mouse/keyboard layout, but with an extra on-screen hint or two for the digital actions "back" and "enter location". The player can still bring up the modal menu from before, but they're encouraged to just use the "move finger cursor" analog action to directly click on the menu they want, and they can either click on a location to move there or use the "move location" analog action (presumably bound to an analog stick or touchpad).

Here's what the config looks like:

Very similar to the "Menu Controls" action set, and using the same stacked trackpad+gyro mouse motion.

And remember that special code I had to write to get proper diagonal motion out of the DPAD? The Steam Controller Configurator has an out-of-the-box solution for that, too!

This is the DS4 configuration for the map screen. Notice that I've assigned the entire DPAD to a single analog action, "Move Location." The Steam Controller Configurator processes the DS4's raw DPAD input and sends it to my game as an analog vector, and now I don't have to add any special logic to my game to properly handle diagonal motion.

Battle

This is where things get heavy.

Mouse/Keyboard

Since this is a tower defense game, none of your characters can move. Your main character starts in a fixed location and defends herself by summoning and placing party members around the map. To do this, you click a character class tab, then a character button, then the spot you want to place them on. Advanced players can use the QWERTY hotkeys to select class tabs and the ASDFGH hotkeys to select a character.

You can also cast spells by clicking a spell button and then clicking somewhere on the map. Advanced players can select spells with the 123456 hotkeys.

If you've read my previous article, Optimizing Tower Defense for Focus & Thinking, you'll remember that one of my key design principles is total information -- nothing of strategic value should be hidden from the player. That said there's only so much room in the UI, so a lot of information you only need occasionally is stashed away under tooltips. For instance, mousing over any of the enemy waves will show their specific stats:

And if you've ever played an RPG with confusing icons or ambiguous statistics you'll be pleased to know we explain all that stuff for you whenever you mouse-over a stat icon anywhere in the game:

Finally, time control is a key skill in Defender's Quest. High-level players need to be able to pause the game at a moment's notice, as well as adjust the playback speed. In casual play it's sufficient to click on these, but high-level players prefer to use hotkeys (SPACEBAR to toggle pause, "-" and "+" for speed up / slow down)

Naturally, all these hotkeys (and more) are configurable.

This is already a pretty tall order for controller-style input. Several key challenges stick out:

How do we handle all this with a mere gamepad?

Conventional Gamepad

First, I bound one of the analog sticks to move the finger pointer around the map and the A button to "click" it. It wasn't perfect, but after processing and filtering the raw analog motion I was able to get acceptable results (the ability to slow the game down helps a lot with this). As a convenient backup, I added "tile movement" that moves the finger cursor one whole square at a time, and bound it to the DPAD. After all, one of the most frequent actions is placing defenders, and they can only be placed at regular grid intervals.

So far this was working out pretty well, but there was just no way I could get my poor XBox 360 pad's analog stick to work as a proper mouse substitute for clicking any random button on the screen, so I bit the bullet and started adding special gamepad-only navigation menus to the battle screen.

To start with, I broke battle navigation down into two separate modes - "free" movement for waving the mouse-like finger cursor around the map, and "menu" movement for discretely jumping the fixed finger cursor around various menu options with up/down/left/right actions.

So when you first start a battle in gamepad mode, you see this:

The button hints tell the player what basic shortcuts are available, but this menu can also be navigated by moving the finger cursor around and pressing A.

If the player moves the analog stick to the left, they will "detach" the fixed-menu finger cursor and turn it into the free-floating mouse finger cursor (note the tilt).

https://www.fortressofdoors.com/content/images/2017/01/dqdx_finger_detach.gif" width="560px"> Your browser does not support the video tag.

If you move the analog stick back to the right, the free-floating cursor turns back into a fixed cursor that moves by jumps:

https://www.fortressofdoors.com/content/images/2017/01/dqdx_finger_attach.gif" width="560px"> Your browser does not support the video tag.

If you move the finger pointer all the way to the left, it will likewise revert to menu cursor mode and "stick" to the enemy waves, so that you can properly inspect the tooltips:

https://www.fortressofdoors.com/content/images/2017/01/dqdx_finger_tooltip.gif" width="560px"> Your browser does not support the video tag.

The real tricky part with all this was tracking all the nuances of input mode state, making sure all the menus were properly nested and would respond in the right way when you pressed the back button, and finally making sure that randomly clicking on stuff with the actual mouse while you were doing all this didn't make the universe explode. (When mouse-like motion is detected, regardless of whether it's a real mouse or the SCAPI, the game shows the free floating cursor and hides the fixed cursor, and you can click on anything you want anywhere on the screen).

Thankfully, most menus didn't have to be redesigned in terms of visual layout; for the most part all that was necessary was some extra logic, and a few extra gamepad-only overlay menus with button hints.

The one exception to this was the "select defender" screen. Here's the mouse and keyboard version:

There's a lot of information crammed in there and a lot of important buttons to click, not to mention two whole menus full of tooltip-able statistics. So this did require a special redesign for when the player is using a gamepad:

I had to brutally crop the character portrait to make room, but this layout is fully navigable with simple up/down/left/right menu commands (as well as the mouse), and provides all the same tooltip information as the conventional layout.

Steam Controller controls

As we saw earlier, the Steam Controller didn't actually need as much special-case logic to get things working. A fairly decent "mouse imitation" mode could have been rigged up just by switching my hotkeys over to actions and exposing them to the configurator, and then let the player just point and click as usual. Most of the special-case logic I beat conventional controllers into shape with would have been unnecessary. Of course, this laziness would come at the cost of somewhat awkward mouse-pointing for XInput users.

Since I had already gone to all this trouble to add a gamepad mode, I decided to go for my "best of both worlds" strategy with the Steam Controller API.

A couple of things to unpack here. First, we can see that the "Move finger cursor" analog action is bound to both the gyro and the right touchpad in the "stacked touch + gyro" method I use in the other two action sets. Note that there's another "Move finger cursor" action bound to the joystick, but it's "Move finger cursor + navigate menu."

Both actions will move the mouse finger pointer around, but the latter replicates the "sticky" behavior for navigating menus via jumps that I used in the conventional controller setup, just to give players some options.

This is an excerpt from the VDF file that defines my Steam Controller API action -- as you can see, the input method for both of these actions is absolute_mouse. However, there is another input mode, joystick_move, and it's important to understand the difference because it's easy to use the wrong one.

Valve's Scott Dalton gives a great explanation:

So basically, use absolute_mouse for all your analog values and it will be flexible enough for any physical input (including joysticks), and you can save joystick_move for cases where you want analog output that specifically behaves like a joystick.

Going back to the configuration screen, most of the bindings are pretty straight-forward hotkeys for various digital actions, so I'll skip those and only point out a couple of special notes.

Look at the right bumper, notice it's binding is "LEFT MOUSE, Spell repeat." What's going on here? Opening the settings for that input shows this:

First of all, there are two actions bound to this button via "activators." Activators are programmable triggers that the player can set. In this case, a regular press does one action, and a "long press" does another (and yes, you can specify what counts as "long").

So what is this "LEFT MOUSE" action? Well, it's not actually an action, it's a direct physical input simulating a real left mouse click. The native API not only lets you communicate directly with the game via actions, it also lets you do all the same things you can in legacy mode via input re-mappings. This is useful for covering weird edge cases. In my case, I couldn't quite simulate true low-level mouse clicks in-game, and I had layers and layers of old code that was naively checking specifically for low-level mouse clicks. Rather than keep fighting with my code I just directly mapped the right bumper to a real mouse click and BAM, my game couldn't tell the difference and everything worked perfectly.

Valve has since told me they've added the ability to send true mouse click events along with a native digital action, so this trick probably isn't necessary any more, but I still wanted to include it in this case study to showcase the broader feature.

Here's how you'd do it natively, though. My digital battle actions look like this:

Note the last entry in the list. After the localization token there's a comma, a space, and mouse_button LEFT. This pipes that specific input along with the digital action signal "battle_click". So if I wanted to replace the legacy binding in my config with a native solution, I'd just assign this new digital action instead and the behavior would be identical. In my specific case I only need the mouse click, so I don't even have to change the game's code -- I simply leave the new "battle_click" action unhandled.

Okay, so we've covered the LEFT MOUSE bit. What's this "long press" thing about?

The "long press" activator is set to go off after holding the right shoulder button down for 0.150 seconds (you can change it by moving the "Long Press Time" slider). In my case this triggers the "Spell repeat" action, which tells the game to keep casting the currently selected spell if it's available.

Some context: in Defender's Quest, the player can cast a "Lightning" spell that does direct damage to a single enemy at the cost of PSI energy, and there are certain popular strategies where a player wants to spam the spell for several seconds or more. This strategy doesn't break the game or anything, but without a "spell spam" button it wrecks peoples' hands from constant clicking. With this action, a player can cast the spell with the right bumper, and then hold it down to keep spamming it.

In my case all of the "spell repeat" logic is handled by the game code, but that's only so I can cover conventional input. You'll notice that the player can rig up their own "hold to repeat action" (aka "turbo") right from this screen -- all the game developer has to do is expose a single digital action to the API; the configurator can do the rest all by itself.

Finally, let's talk about this interesting-looking input:

The left touchpad is normally bound DPAD-style to four digital actions (tile movement), but I've set up a special setting called a "Mode Shift", which uses the left shoulder button together with the left pad to activate a "Radial Menu."

To see this, first you inspect the left pad's bindings:

Then you click on "advanced settings":

From here we can see that I've added a "Mode Shift" to this input.

And here's the final settings for the radial menu:

When the left bumper is pressed, and the left pad is clicked, then a radial menu will appear.

Radial menus arrange a set of digital actions in a single easy to use selection scheme. In this case the actions are selecting spells 1 through 6.

Note that the "Menu Activation Style" is "Touch Release/Modeshift End" -- this means that when I stop pressing the left bumper, or let go of the left pad, the radial menu will trigger the selected action. Also note there is a "click action" assigned -- pause.

So now I have an easy way to select from various spells, and the very act of bringing up the menu pauses the game for me, too.

The best part is that all of this UI is driven entirely by the Steam overlay and the Steam Controller Configurator. The only thing I needed to add in-game was six digital actions for selecting spells 1 through 6. Even the icons you see here are driven by the API via the Steam Overlay.

The configurator has a large library of icons to use, but a developer can supply their own by including a folder called "TouchMenuIcons" in the root directory of their game. Furthermore, the configurator can control the size of the radial menu, as well as its on-screen position, opacity, and size.

As a special note, I've found it convenient to create a totally unused digital action called "Nevermind" and bind that to the center of the radial menu, and give it an "X" icon. Since "Touch Release/Modeshift End" commits you to an action as soon as you close the radial menu, this creates a safe "dead zone" in the middle of the radial menu to exit and do nothing.

Summing Up

The Steam Controller Configurator can do far, far more than what I've briefly glossed over here, and I'm sure I'm not even using it in the best possible way. I do hope that this case study based on a real-world example brings some of its lesser known strengths to light and inspires you to do cool and amazing new things with it.

I know the whole interface looks complicated at first glance, but I want to underscore that in terms of the code you need to add to your game, it's often less work to add support for the SCAPI than it is to properly handle all the edge cases and limitations of conventional input. And even if your game is very mouse-and-keyboard centric, you'd be surprised how well you can make Steam Controller Devices work with your game just by exposing a few actions to the API and playing around in the configurator.

It's my dream that all of PC gaming will eventually work as smoothly as the Steam Controller API does. It does have a few kinks left, but Valve has been incredibly dilligent with updates and shows no signs of slowing down. I can't wait to see what the future holds.

Useful links

If you want to learn more about the Steam Controller initiative, I highly recommend the /r/steamcontroller subreddit as well as their associated Discord channel.

I can also recommend these three open-source projects: