Steam Dev Days: Steam Controller

This is a record of the transcript and slides of a talk originally given at Steam Dev Days, October 12th 2016. For more details, please see SteamDevDays.com

Title: Steam Controller
Speakers: Jeff Bellinghausen (Valve) & Lars Doucet (Level Up Labs)
 

Hello. I’m Jeff Bellinghausen, I work on the Steam Controller team at Valve. Today Lars and I are going to discuss the benefits of integrating with the Steamworks Controller API, and how it can be used to simplify your games input code.

Before we hear Lars dive into the API itself, I want to give a quick overview of how the Steam Controller is doing in the marketplace.

One of the reasons we liked using trackpad technology when we were designing the steam controller is that they can be easily virtualized through software, allowing us to expand the controller’s functionality over time. It’s now been almost exactly a year since we shipped the first Steam Controller preorder units, and we’ve since pushed over 72 updates to Steam that specifically target the controller functionality.

Some notable updates added “activators” that let players do things like assign an action to a long press of a button,”action sets” that let players define multiple configurations that they can cycle through on the fly, and on-screen radial menus that can map up to 20 different actions to a single pad.

Soon we will be shipping an update that adds full configurability to controllers other than the Steam Controller. This means that players can pair their PS4 controller directly to their PC and use all the configurability options available to the Steam Controller, including use of the PS4 touchpad and gyro.

(plays video of DOOM footage -- I will update this with a link if it becomes available online)
 

We plan to continue actively extending the feature set for all controllers.

Customer satisfaction is very important to us, so we like to actively monitor community reviews. We’re happy that the Steam Controller has been well received by customers, and currently holds an 81% “very positive” rating on Steam, and a 4.2 out of 5 star rating on Amazon. In addition there are several active forums dedicated to the controller that we monitor to help guide our improvements.

Players have shared over 1.2 million game configurations since launch.

We announced in June that we had sold 500,000 controllers, and we’re now on track to sell a million by early next year.

We’ve been actively including the controller in Steam promotions, and have noticed similar response to what we see with software, including a persistent rise in baseline units after the sale.

This graph shows the full year of Steam Controller sales, with controllers sold as part of a software bundle in different colors. You can see that the visibility boost increases sales of the bundled controllers as well as standalone controllers.

We’re working on ways to improve discoverability of games that play well with the controller. Now that there is a significant installed base of controller users we are working on ways to promote games that they may be interested in. One of the more direct methods is to present customers with a ‘most popular with Steam Controller’ games list. Titles on this list will be easily discoverable for both new purchasers as well as existing customers. We will also continue to run focused sales that highlight hand-picked controller-friendly titles.

Since their launch, Controller and Link sales have been limited to the US, EU and Canada. We’re now expanding that to Eastern Europe, SE Asia, Latin America and Oceania. The first of these new territories are already online, and the rest will be there over the next few months.

Now I'd like to move on to the discussion of integrating the Steamworks Controller API into your title and what that means to you as a developer.

Here are a few reasons why using the Controller API means less code in your game, not more.

If you use the built-in Steamworks controller configuration UI, you don't have to build a bunch of controller-specific UI in your game.

This means that your players can fully configure their controller, and sometimes in ways you can't anticipate. We’re often surprised at the novel setups players come up with and I think you will be as well.
 

We’re adding support for all major controllers, so you can worry less about drivers and more about your game. We’ll even provide ready-to-use in-game hint glyphs.
 

Once you've done the integration, you can benefit from future Steam updates, whether they be software configuration options or new hardware support, all without having to update your game.
 

So hopefully today's session will be enough motivation for you to consider integrating the API into your game …
 

But if you can’t for some reason, I want to give you a few guidelines that will ensure customers can still have a positive experience with a controller.

First of all make sure that the game can handle simultaneous input from a gamepad and a mouse. Most preferred Steam Controller configurations use the gamepad for most functions but control the camera with the mouse. There are two things to look out for here.
 

First, make sure your engine supports character movement via the gamepad stick and camera movement from the mouse. Some games will lock out one form of input whenever the other is active.
 

Second, make sure your in-game hints are keyed off of the gamepad and keyboard, but not the mouse. This will prevent the on-screen hints from flickering back and forth between gamepad and keyboard hints.
 

Once your game is working well with the gamepad, make sure you have a good default configuration set on the Steam Partner website. This can be as simple as selecting one of the built-in templates, if they’re well matched for your title, or, you can create a custom configuration by hand and specify it by the Workshop File ID provided by Steam.

That’s it for my introduction, now I’d like to hand over the podium to Lars so he can talk about the API in detail.

 Thanks, Jeff.

Real quick, who am I and why should you care? Unlike Jeff, I don’t work for Valve. Like many of you, I’m an independent game developer, I co-founded Level Up Labs, we made a game called Defender’s Quest that did pretty well.
 

I also contribute regularly to Open Source projects, most notably the programming language Haxe, the multimedia library OpenFL, and the 2D game framework HaxeFlixel.

I also wrote some Gamasutra articles you might have read that one time.

We've been busy adding native Steam Controller support to Defender's Quest and we'll be shipping that real soon.

Speaking of Gamasutra articles, the inspiration for my part of the talk goes out to Zach Burke, who wrote The 5 Golden rules of Input. I don’t have time to cover it in depth except to list those five rules for creating a great PC gaming input experience, and here they are:

So with our destination firmly in mind, let’s survey various approaches people take to get there.

Hard coded physical inputs – IE, everybody’s input logic in their first game maker game.

This is when you directly check for specific inputs like WASD, ENTER, the “A” button on (specifically) an Xbox 360 Controller (specifically) in controller slot 1.
 

Not only can’t the player rebind the keys, the game only support the exact devices that are checked for, which in turn means you only have one set of input icons that assumes a specific device.This is very common to see with beginners, but I’ve actually seen it in some big commercial games, too.

So we’re good programmers, we know to abstract things. Instead of checking for a specific model of controller in a specific slot, you build a backend that hides all that complexity and tells if you if any connected gamepad of any model has the bottom face button (A for XBOX, X for Playstation, B for Nintendo, etc) pressed. While you’re at it, you can add some abstraction for your action inputs so you check “move_right” rather than the “D” key, etc. This makes things input rebinding easy.

Sometimes, developers paint themselves into a corner and find they have no way to support gamepad input. This famously happened with Edmund McMillen and the original Binding of Isaac – a Flash game made with ActionScript 2. I’m not trying to bash Edmund by any means; point is that he had no option for gamepad support except to use an external utility like Joy2Key, which runs outside your game, listens for gamepad input, and then generates system-wide keyboard inputs that your game then picks up. As you can imagine, there’s latency, instability, and general fiddliness when you use this approach.

So those are some of the general approaches we tend to see with input. Let’s look at some specific problems that tend to arise.

The “Sea of Buttons” phenomenon is really common with PC games. One big global list o’ inputs and every action has its own input. This approach takes keyboard real estate for granted, is hard to adapt to gamepads, and can often (but not always) be a sign of lazy design.

The typical solution is to consolidate several actions into a few context sensitive inputs.

But in pathological cases, that can lead to the opposite problem – “too much context!” We’ve all played games with ambiguous input where the same button that means “kill enemy” is also used for “talk to a civilian.” Then some tiny detail like not facing exactly the right direction switches the context and bam! You just roasted the Mayor.

The “Sea of buttons” isn’t limited to keyboards. Now this image is actually a joke, but I’ve seen plenty of real-world games with crammed-full layouts like this.

So, to summarize, the common input problems we still see in modern games include:

I should also mention instability and poor handling of multiple devices – what happens if you yank out a controller mid-game and then plug it back in? Do Logitech controllers play nicely alongside Playstation and Xbox? And so on.

So “actions” are a better way to think about input that largely solves these problems.

The whole idea behind actions is you totally double down on abstraction. The backend is completely responsible for directly talking to your input devices and just doing the right thing, and your game doesn’t care at all about raw input.

Because you don’t care that A was pressed. You care that Mario JUMPED. Stimulus and response are totally separated. The Steam Controller takes this model even further – you literally never see inputs, ever. You ONLY get actions.

There’s two kinds of actions – Digital and Analog. Digital actions are things like “Jump”, “Shoot”, “Open Door” – ie, a binary on/off state. Analog actions can report a range of values, and can have up to two axes -- things like “move”, or “aim”.

Most analog inputs can be configured to generate digital actions – like turning one of the touchpads or the stick into a DPAD, for instance. Also some “analog” inputs have an underlying digital input –the touch pads and stick can be physically clicked in.

I need to be super clear that you never get inputs directly.  The Steam Controller API only ever tells you “this action just happened,” and if it’s an analog action, “here are the analog values.”

When prepping for this talk I asked people what they wanted to know, and I got questions like, “how do I activate the gyro?”

As the developer, you never activate the gyro, or the analog stick, or the touchpads, or anything. The player is totally in control of what inputs are assigned to what actions.
 

But YOU define what those actions are, how they behave, and when they’re active. Also, you can – and should – provide a default, official input configuration for your game, and that’s where you can be as specific as you like that A should be jump and the right touch pad should move the camera with this much acceleration and with an inverted Y axis, blah blah. But players can – and will – create their own, and your game will be none the wiser.

Actions go along with Action SETS. Rather than a “sea of buttons”, you divide your various actions into discrete sets organized around different modes or settings. There’s no need to have dedicated always-on inputs for “Battle” for instance, instead you should just have a “Battle” action set that becomes active when you enter a Battle. Likewise you can have “Driving” vs “Walking” vs “Hacking”, or whatever makes sense in your game.

As you can see here, I divided up Defender’s Quest’s inputs into “Battle”, “Map”, and “Menu” controls.

Action sets go beyond just being a convenient abstraction. You can actually assign action sets to controllers on an individual basis – and you can have up to 16 steam controllers connected to a single PC. So if you’ve got a couch co-op game with diverse roles, you might assign different actions for each player class, or whatever. Of course, you can set all controllers to use the same action set at once, with a single command.

Steam has a set of guidelines for “full controller support” which is geared around a seamless experience for big picture mode.

You’ll notice a lot of these overlap the “5 Golden Rules of Input” I mentioned at the beginning. Basically the idea is that the player should be able to launch your game in Big Picture mode and everything should just work with nothing but the Steam Controller, but they should still be able to use other devices if they want.
 

Most importantly, your game must be oriented around actions, not inputs.

But let’s go ahead and talk about what physical you have available. What I’m showing you today is just what’s available now. The beauty of this is that even though the hardware has shipped, Steam can update the firmware any time and add new features, as well as update what the configuration software lets players do.

The bottom line is that players are in complete control of the actual inputs. But you should know what they are.

We’ll start with the two big circular haptic touchpads on the front. The left pad has a recessed “DPAD”, but is otherwise 100% identical in function to the right pad. The player can use a pad as a full analog input that sends both X & Y information (your game will show the “swipe” glyph), it can be treated like a DPAD with a unique digital input assigned to each quadrant (your game will show one of the four “Analog pad Dpad” glyphs), the pad itself can be physically clicked in (the “click” glyph), or it can detect whether it’s merely being touched by a finger or not (the “touch” glyph).

The steam controller also has conventional inputs – an analog stick and some face buttons. The big “Steam” button in the center, much like the XBOX button on a 360 controller, seems to be reserved by the Steam Client. You have Start & Select buttons too. The Analog stick can be configured to be pure analog input, it can be treated to send digital direction events, DPAD style, and it can be physically clicked in.

Next, you have digital shoulder buttons or “bumpers” on the top, and you have analog triggers below them that have a physical, digital “click” when you pull them in all the way. Lastly, you have two inputs unique to the steam controller – “grip” paddles on the left and right, where your bottom three fingers are holding the controller. These are simple digital buttons.

Finally, you have gyro controls, which not everybody knows about. The Steam Controller has a full motion sensing gyroscope in it, detecting pitch, yaw, and roll. This has become a pretty popular input these days, particularly for aiming in FPS games. The Gyro is obviously an analog input.

The hardest thing to adjust to is learning to communicate INDIRECTLY with the steam controller. For me this required me to redesign my game’s inputs, but as a side effect, it made it easier to enable conventional controller support, which also helps when porting to consoles.

The other thing to mention is that this is a C++ API, so if you’re using some other language you should make sure you have a native extension. If you’re already shipping a game on steam that supports e.g. achievements, this shouldn’t be a huge deal – I added Steam Controller support to the Haxe native extension, for instance.
 

The API is based entirely around integer “handles” for controllers, actions, action sets, everything really.

Lastly, the API encourages you to poll continuously rather than setting callbacks once and only responding to state changes. The API clearly marks which methods should be called, and notes that they are “cheap.” If you’re using a non-C++ extension via native extension, it behooves you to check that your extension is not expensive or leaky when it comes time to call these methods.

You’ll need to start by setting up a VDF file (a quasi JSON-like format Valve likes to use) that defines what actions your game has. You must specify which actions are digital and which are analog,

And for analog actions, whether they’re mouse-style or stick&gyro-style. All configuration beyond this is left entirely to the player.

You can use localization tags to apply any labels you want to anything, but from what I can tell, action sets must have unique string identifiers, and so must actions.

This means you can’t have the action “up” in two different action sets – you might want “move_up” and “menu_up”, though you can of course apply the user-facing label “Up” to both via a localization tag.

Digital actions have no properties except their name. With analog actions you can specify what kind they are, and this changes the kind of data you get as well as the input options for the player.

As of right now, there’s only two “modes” – should it act like a joystick, where it’s all about X/Y distance relative to a fixed center, or should it act like a mouse, where it’s all about absolute X/Y distance from wherever the pointer was last frame. Additionally, with mouse-like movement you can set the “os_mouse” flag to true, which will send the analog motion not only to your game but also directly to the operating system’s mouse cursor.

You don’t get to control whether it acts like a trackball, or whether it has acceleration, or whether the Y-axis is inverted – that’s entirely in the player’s hands.

The first thing you’ll do is call GetConnectedControllers() from the C++ API. This returns a ControllerHandle_t value which is just a unique 64-bit unsigned integer. You’ll want to store that and associate it with something a little easier to keep track of like, “controller 1” or a 0-indexed array or something.

NOTE: If you’re using the interface from a language other than C++, you might have difficulty getting the uint64 value back and forth across your API bridge. In my case I built an extension in Haxe, and I found it easiest to just leave the raw uint values in a data structure on the C++ side, and from Haxe I would just ask for controllers 0-16, which resolves on the other side to the proper uint64 handle before querying the Steam API.

Also note there is a STEAM_CONTROLLER_HANDLE_ALL_CONTROLLERS that’s defined in the API, that you can use anywhere a controller handle goes.

The next thing you’ll do is call GetActionSetHandle() for every action set you have in your game. You’ll pass in a name and get back a value.

There are different handle types for digital actions, analog actions, and action sets, but they’re all uint64’s underneath.

NOTE: each of these functions takes only ONE string as input – ie you get actions by their name alone, not their name and their set. This is why action names must be unique!

When you change action sets, you should call ActivateActionSet() to update the steam controller. This is a “cheap” call and you can safely do it constantly, so you’re less likely to have bugs if you just spam this constantly rather than only doing it exactly once, whenever the in-game set changes.

To actually get action data, you call GetDigitalActionData() and GetAnalogActionData() continuosly. All actions have a “bActive” boolean that tells you whether the set is currently activated for the given device. Other than that, digital actions return a simple true/false, and analog will give you an X/Y value. Note that “joystick_move” and “absolute_mouse” will change the kind of x & y values you get.

Here’s a quick quality of life tip – just getting an action’s raw state often isn’t enough, you want that in the context of what’s been going on. So in the game framework I use, HaxeFlixel, we added an additional layer where for every input or action, we compare its current value to its previous one, so you can check not only whether it’s currently pressed, but also whether it was JUST pressed. This only takes a few lines of code to write, but it massively improves the programming experience and gives you more action states to use in your game.

Although we don’t have direct contact with inputs when using actions, we do have access to the glyphs. Calling GetDigitalActionOrigins() / GetAnalogActionOrigins() will return an array of Enums (ie, integers), which correspond to every possible user input the steam controller can perform. Valve says you should call this method continuously rather than just calling it once and caching the result, because the user can change their configuration at any time.

Because I was using a native extension and having trouble sending array data back over the bridge without getting mangled, I kind of compromised here. My extension packages the result as a string (yes I know, I know), which is a lot of allocations to do 60 times a second, but isn’t so bad if it’s only say, once a second. So if you’re in the same boat, I agree with Valve you should poll continuously, but since the user changing their config is a bit of a rare event, you can probably get away with only listening for input origin changes regularly but intermittently rather than every single frame. Worst case scenario, if you poll e.g. every 1 second, there will be at max a 1 second wait after they close the steam overlay before the glyphs update, which seems tolerable.

Remember – Valve’s implementation is fast, but your native extension might not be! If you’re a Unity user, be sure to profile & test your steam controller module!

Valve provides their own set of glyphs, which are super professional and nice and you should use them. In my case I found the blue tint clashed with my game so I came up with a more neutral palette that also matched the style of the rest of the glyphs I was using, based on Xelu’s exhaustive gamepad icon set from OpenGameArt. I’ll provide a link to my glyphs at the end of the talk.

The Steam Controller has experimental support for haptic pulses, which is similar to “Rumble.”

Unlike other controllers, the Steam Controller doesn’t have the typical “rumble” motors, it instead relies on the existing haptic feedback in the touchpads. You have two very low-level functions to trigger haptic pulses. Whereas with other controllers you call a function like rumble(STRONG/WEAK,duration), with this you just send an on/off signal for a very short amount of time – the max is like 0.03 seconds. So you’re meant to create patterns by stringing calling this with repeat parameters to control the texture of your pulses.
 

The pulses themselves are rather mild compared to conventional rumble so don’t expect anything earth-shattering. That said you can do cool stuff with them – one person even made them play music!

I should note that you have API calls you can use to bring up the Steam overlay both for text input and to let the user rebind their controls. If you’re only shipping your game on steam, you can use the steam controller binding panel instead of writing your own; you can still write your own for conventional input, that’s up to you.

But Valve strongly recommends you invoke the text input API call when in controller mode. Not only is this a really good soft keyboard, it’s also got full language support, so your Korean, Japanese, and Chinese players won’t be cursing your home-rolled text input system for not having full IME input. In fact, using Steam’s text input overlay when you’re in gamepad mode is required if you want your game to claim “Full Controller Support” status.

So a new feature that’s coming is support in the Steam Controller API for other game controllers. The first one up is the PS4 controller.

Believe it or not, when you use the PS4 Controller through the Steam API, it’s exactly the same as a Steam Controller. You make the exact same API calls, you only get actions, not inputs, and the Steam API takes care of everything.

Now, why start with the PS4 controller? Well, not only is it a really nice, high quality controller, but it’s also got a gyro and a touchpad, so it’s got a lot of overlapping functionality with the Steam Controller. Also, existing native support for the PS4 controller on the PC is a bit weak; in this case Steam itself is communicating directly with the device so everything that’s nice and reliable.
 

Similar support for other controller models is coming, but Valve isn’t ready to announce any devices or dates for those just yet.

This is what input configuration looks like for the PS4 controller when used through the Steam API. Again, you won’t see this at all from your side of the API, it’s only the user who sees it. However, you will get access to the glyphs through GetControllerActionOrigins(), and as you can see they’ll be PS4 specific.

So what are the benefits of all this?

First and foremost is you now don’t have to write your own key rebinding interface if you don’t want to – as long as you support the Steam API, the user just pulls open the overlay, changes their inputs, goes back to the game, everything just works. And if you’re using something like Joy2Key, this rescues you from that nightmare. It also removes the need for a key config launcher.

Next, your code gets cleaner and easier to manage. Even better, the Steam API handles multiple devices connecting and disconnecting, so you don’t get null pointer crashes and whatnot if something is suddenly yanked away, you just get no input from that device.
 

Best of all, it’s easier to achieve the 5 golden rules of input.
 

Now, extra credit – cool stuff that you can do with this new paradigm if you’re resourceful. This will require you writing a bit of your own actions backend in your own code, but I expect to see this sort of thing quickly standardized for all the popular middleware like Unity and Unreal.
 

For one, if you’re taking your game off-steam, having a thin actions layer is the natural place to polyfill what the Steam API does with your own actions abstraction. For instance, I’ve built such a backend for HaxeFlixel, the open source game framework I rely on.
 

But on Steam or off, actions make it easier to write a proper replay system. If you’re manually saving inputs like “W was pressed at time XYZ” that’s a lot more error prone than “Mario JUMPED at time XYZ”. Next, you can more easily set up AI contrOl by using the same input logic you’re using for your players, but you just spoof the events yourself.  Also, as I mentioned before, it’s good to add a thin layer of input state tracking to your actions so you can track “JUST pressed” vs “IS pressed” and so on.

So just to sum up what we’ve talked about, the most important thing to understand about the Steam Controller is actions & action sets.

You’ll need to get more comfortable with indirect input where you don’t have “isKeyPressed(W)” sloshing around directly in your update loop.

You have to get used to not knowing, or caring, about what the actual inputs are.

Other controllers are on the horizon, with the PS4 being the first among them.

And yes, we’ve got all this whiz-bang input stuff, but let’s remember what it’s all in service to: a great user experience. These are all just tools, if we miss that mark, then nothing else matters.

------------------

Final notes:

API Documentation (requires SteamWorks developer account)

https://partner.steamgames.com/documentation/api

Official Steam Controller Glyphs (see "Steam Controller" in the API docs)

Lars's custom controller Glyphs (via OpenGameArt):

available here: http://opengameart.org/content/free-steam-controller-glyphs-pack
based on Xelu's work: http://opengameart.org/content/free-keyboard-and-controllers-prompts-pack

--------------

Open source projects mentioned:

Haxe Toolkit
http://haxe.org/

OpenFL

http://openfl.org

HaxeFlixel

http://haxeflixel.com/

NOTE: The Actions API for HaxeFlixel has not yet been merged into the main branch as of this writing. Coming soon!