Porting mobile game UI to microconsole controllers

I have a great advantage of working for both worlds: developing mobile games as part of Jenots team (http://jenots.com) and working on a Lua based crossplatform 2d mobile game engine Gideros (http://giderosmobile.com) as my main workplace. Therefore, when the opportunity comes to add new features or support of new platforms/libraries to Gideros SDK, I can always test and try them out in real games, to come up with best API design and solution to integrate them in Gideros engine.

Great example of such situation was adding support to OUYA. While matching controller buttons and sticks, and integrating IAP support is the easy part, the most hard part was to come up with the solution, to help Gideros users easier port their existing mobile games to OUYA console.

The core gameplay can be quite unique to any specific game, and its binding should be thought thoroughly by developer’s themselves, there was nothing we could really do about it. But there are parts, that repeat in almost any mobile game, in the start screen, in the options, when selecting levels, etc. There are scenes, with predefined areas, which user could click/tap to interact with the game. So what we could do, was to automate the porting of these quite similar in functionality scenes, by providing an additional layer, accessible only for controllers.

And so the revolution begins.

As usually my experiment base was a game we released some time ago, called Mashballs (http://jenots.com/mashballs), which is a simple physics puzzler, with a goal to mash all target balls and with very live and dynamic gameplay, of manipulating the ball, by using “magnetic” force from your own finger tips.

If you check out the game, you will see that there is only one single scene, which will have unique game specific interface - the level scene itself. Other ones, like start scene, options scene, level select scene, etc are quite similar from the usage point where they only have some buttons defined with the actions attached to them.

So from the developer point of view, Gideros scene usually consists from a scene class and objects attached to current scene hierarchy.

--scene by default is inherited from Sprite
startScene = Core.class(Sprite)

--scene constructor
function startScene:init()
	--let's create couple of simple TextFields
	local startText = TextField.new(nil, "Start")
	--set their position
	startText:setPosition(500, 300)
	--add a mouse/touch event to it
	startText:addEventListener(Event.MOUSE_UP, self.startGame, self)
	--add this text object to the scene, so it will be displayed on it
	self:addChild(startText)
	
	--and create couple of more buttons in similar style
	local optionsText = TextField.new(nil, "Options")
	optionsText:setPosition(500, 400)
	optionsText:addEventListener(Event.MOUSE_UP, self.gotoOptions, self)
	self:addChild(optionsText)
	
	local aboutText = TextField.new(nil, "About")
	aboutText:setPosition(500, 500)
	aboutText:addEventListener(Event.MOUSE_UP, self.gotoAbout, self)
	self:addChild(aboutText)
end

And of course we also need to define method, which will be executed on this button clicks.

function startScene:startGame()
	--go to level scene
	sceneManager:changeScene("level", 1, SceneManager.flipWithFade)
end

function startScene:gotoOptions()
	--go to level scene
	sceneManager:changeScene("options", 1, SceneManager.flipWithFade)
end

function startScene:gotoAbout()
	--go to level scene
	sceneManager:changeScene("about", 1, SceneManager.flipWithFade)
end

Developing Controller Layer

Now the best way to integrate the controller layer seemed to use it as the scene itself (as all objects will be added to the scene anyway), so theoretically we could create a class, from which each scene could inherit properties and use them to add controller support to current scene.

OUYALayer class

--create OUYALayer class inherited from sprite
OUYALayer = Core.class(Sprite)

--ouya layer constructor
function OUYALayer:init()
    --if ouya plugin is not activated
	if not ouya then
		require "ouya"
	end
	--flag if layer is disabled or not
	self.__disabled = false
	--hold all clickable objects here
	self.__objects = {}
	--currently selected object
	self.__selected = 1
	--previous object
	self.__prev = 0
	
	--add listeners
	self:addEventListener(Event.ADDED_TO_STAGE, self.__added, self)
	self:addEventListener(Event.REMOVED_FROM_STAGE, self.__removed, self)
	self:addEventListener(Event.ENTER_FRAME, self.__frame, self)
end

Now we will need to listen to events when the scene is added or removed from the stage so we could add or remove ouya specific events.

What we would want to do, is to listen to D-pad and Left stick events, so we could change selected elements on user input, we should also listen to O button to emulate the click on selected element and additionally we can listen to A button and provide a specific event or call a specific method to go one scene back.

function OUYALayer:__added()
	--add ouya specific events
	ouya:addEventListener(Event.KEY_DOWN, OUYALayer.__handleKeys, self)
	ouya:addEventListener(Event.LEFT_JOYSTICK, OUYALayer.__handleJoystick, self)
end

function OUYALayer:__removed()
	--remove ouya specific events
	ouya:removeEventListener(Event.KEY_DOWN, OUYALayer.__handleKeys, self)
	ouya:removeEventListener(Event.LEFT_JOYSTICK, OUYALayer.__handleJoystick, self)
end

Adding object to our layer

Now let’s define a method to add objects to our internal table of clickable buttons which is stored in self.__objects.

function OUYALayer:addObject(sprite)
	table.insert(self.__objects, sprite)
end

Selecting the object

On each enter frame we should check if currently selected object have changed so we could change it appearance as we deem necessary, to make it visually excel, and also undo the previous objects appearance changes if needed

function OUYALayer:__frame()
	--check if layer is not disabled
	if not self.__disabled then
		--check if there is any object
		local sprite = self.__objects[self.__selected]
		if sprite then
--and if current object not equals previous one
			if self.__selected ~= self.__prev then
				--Now we can draw something on our button here
				--using Shapre or modify the appearance of selected button
				--as we wish
			end
		end
	end
end

Handling keys

As we listen to key and stick input events we should also handle them, so let’s define methods to do that.

function OUYALayer:__handleKeys(e)
	--check if layer is not disabled
	if not self.__disabled then
		--if O button was pressed
		if e.keyCode == KeyCode.BUTTON_O then
			--we check if there is any currently selected object underneath
			if self.__objects[self.__selected] then
				local sprite = self.__objects[self.__selected]
				--if yes we retrieve it and check what event is has and dispatch them to emulate mouse/touch events on it.
		--if A button pressed
		elseif e.keyCode == KeyCode.BUTTON_A then
			--we check if our scene has back method, and if yes let’s execute it
			if self.back then
				self:back()
			end
		--then we simply handle directional input from D pad and call specific method
		elseif e.keyCode == KeyCode.BUTTON_DPAD_DOWN then
			self:__goDown()
		elseif e.keyCode == KeyCode.BUTTON_DPAD_LEFT then
			self:__goLeft()
		elseif e.keyCode == KeyCode.BUTTON_DPAD_RIGHT then
			self:__goRight()
		elseif e.keyCode == KeyCode.BUTTON_DPAD_UP then
			self:__goUp()
		end
	end
end

Quite similarly we handle the left stick input.

function OUYALayer:__handleJoystick(e)
	--convert radians to degrees
local angle = math.deg(e.angle)
	--check if we have marked the stick as dirty and if it is almost at furthest point
--by checking the strength property
	if not self.__isDirty and e.strength >= 0.99 then
		--if yes we mark the stick as dirty, as we already received input from it
		self.__isDirty = true
		--and then based on angle we again call direction specific method
		if angle >= 0 and angle < 45 then
			self:__goRight()
		elseif angle >= 45 and angle < 135 then
			self:__goDown()
		elseif angle >= 135 and angle < 225 then
			self:__goLeft()
		elseif angle >= 225 and angle < 315 then
			self:__goUp()
		elseif angle >= 315 and angle <= 360 then
			self:__goRight()
		end
	--but if stick was marked as dirty and no its strength fell lower than 0.5
--it seems stick was put back, and we can again start receiving the input from it
	elseif e.strength <= 0.5 then
		self.__isDirty = false
	end
end

Navigation types

Queue navigation

There are different types of navigation we have tried out, first the queue navigation, where all objects are processed in same order they were added to the self.__objects table. The right and down directions select next objects in the table and left or up directions select previous objects in the table. This is quite easy to implement and it is pretty efficient solution.

Additionally when adding objects to the layer, you can allow providing id of the object, so developers could easier control the order of how objects are selected, not only by order they were added.

Directional navigation

Simple directional navigation would select the closest object by specified direction. For example, in each navigational method (OUYALayer:__goUp(),OUYALayer:__goRight(), etc), we need only to check the objects that are further from our current project. If we are searching for the object on the right, we can retrieve currently selected object coordinates, then iterate through all added objects inside our self.__objects table and select the object with lowest x coordinate, which is higher than current object’s x coordinate.

In this case we don’t mind the other axis at all, and this while in most cases (especially simple grid level select scenes) worked correctly, in some scenes it proved to be somewhat confusing both to players, as moving stick to the right, selector jumped up and down between buttons.

Distance based navigation

 Quite similar to  directional navigation, but in this case we don’t simply select closest object on single axis, but take into consideration both axis and calculate the distance (vector) for the objects that are on the needed side of current object.

This technique proved to work the best and behaved as expected to many players that tested it, but it has one great disadvantage. There might be a situation when, if you have a cluster of more close one buttons, you may have a button, which you will never reach.

Mixed solution

Current solution that we stopped at is to consider both axis, but not simply calculating distance, but more rule based approach. So if we want to move right, we first find the closest objects on the y axis (and not x as you might have expected), we take absolute value to check objects in both up and down directions, then when we find closest objects in some specific threshold (for example taking dimensions of current object into consideration), we check which of this object is closest on the x axis in specified direction and go to it.

If not objects are found on y axis in specified threshold, then we again simply check the closest object on x axis in specified direction.

Applying OUYALayer

Now when we have basics of OUYALayer, let’s apply it to our previous scene.

Step one, we make scene inherit from OUYALayer

startScene = Core.class(OUYALayer)

Step two, we add our buttons to the self.__objects table

self:addObject(startText)
self:addObject(optionsText)
self:addObject(aboutText)

And that’s it, no step three, now you can navigate by them using OUYA Controller and click them by pressing O button.

Here is the end result we achieved with Mashballs game.

Things to consider

Here are some additional things you might consider when implementing something similar.

Handling sublayers

In some cases you will have subscenes, or in scene pop ups, which will overlay your current scene. In this case it will be bad if user could still select the items underneath, so when implementing something similar, you should consider enabling and disabling controller layer. So you could make popup also inherit from OUYALayer and disable the scenes layer activities.

Scrollable views

Some scenes can contain too much information and have scrolling, so on selecting each new object, you should check its position and place is in the center of the screen if possible.

Tweening

If you will have some sort of overlay upon the buttons, to mark the visually as selected ones, you may consider also tweening the overlay between buttons, to provide smoother and more visually appealing transition.

Conclusion

This article demonstrated the key points you can use, when implementing controller layer upon your existing touch/click based games, to easily port them to any controller supported console.

It used Gideros SDK and OUYA microconsole as example implementation, but I believe these key points could be used and applied to many different game engines.

While examples provided here are quite simple, just to show and explain the technique, the end result - OUYALayer class is much more complex and sophisticated to handle different scenarios and needs of Gideros developers, so don’t get afraid if you stumble upon its source code :)