Shiny is our Psychedelic Neck Trainer, where you can play using your own music.
The objective is to destroy moving cubes, by simply looking at them.

As easy as it sounds it can get harder, when the screen is full of moving objects;)

We recently ported Shiny from Oculus Rift to GearVR and are presenting it at GDC using a Samsung Galaxy S6, running on Unity 5.3.

In this article I will talk about things I came across, while porting Shiny. Some of the mentioned methods
are quite general techniques to improve performance, since improving the framerate was the most important task. The GearVR (especially when you also want to run on Galaxy S4) is a lot slower then a powerful gaming PC behind an Oculus Rift.

Actually the port itself went really smoothly, since the game was allready made for Virtual Reality, but we had to improve our performance by a lot!

There was another interesting article on Gamasutra some days ago about bringing a game to GearVR so I recommend reading it as well.

I am aware, that I will explain stuff, which will be considered trivial by some people. However, I prefer explaining things which may be too basic, over not finding information needed, because it is considered to be too easy to write down.
 

General Stuff

- Clean up your project, especially when you are using a newer unity version.

We had the error ““CommandInvokationFailure: Unable to convert classes into dex format. See the Console for details.” caused by an old JDK somewhere in our project which clashed with the now (better) integrated development kit in Unity 5.3. The error could be fixed by finding vrlib and vrplatlib in our plugins and deleting these two files. In general this error means that you have some .jar file twice. It does not even mean that it must be in your project, but can also be in your unity installation. ou can try to find the file mentioned in the error and delete it (once) to solve this error. We did not notice the error earler, since there was no problem when building for Oculus Rift it only occurs, when building vor android.
There might also be some other old relics in your code which are no longer needed.

- Check the (external) tools used.
We used some DirectX 11 particle effects before, which were of course no longer working.
Sidenote: As of Unity 5.3 you can use gradients to set the color over time of a particle. One of the features we used the DirectX 11 particle effects for.
Additional the old Unity Shader we used for bloom was not working on (older) android devices, fortunately the newer BloomOptimized Shader will automatically use a mobile compatible Shader if needed.

- The resolution will be better (especially if you used an old Oculus Rift Development Kit), this means you might need to rescale some UI elements, and you might want to use textures with higher resolution as well. Sometimes even increasing the size of some hitboxes (on far away objects) makes sense, since it will be harder to hit them with your pointer. Additional you can make the used texts a bit smaller without causing them to become unreadable.

- Do not forget your GearVR keyfiles. Keep in mind that you need different ones, when you want it to run on several different Samsung Galaxys. Our game runs on Galaxy Note 4 and upwards, so we needed several keyfiles.

- Adjust the settings (for the music folder) when using GearVR. Before porting to GearVR we used a config file in unity to store the path to the music folder. On android this would still be possible, but a bit annoying since we would need to use the www functionality from Unity.
Fortunately on the Samsung Galaxy there is a standard path to the music folder: "/storage/emulated/0/Music" which we are using now.
In general it can be useful to have yome parts of the code behave differently depending on the platform it is running. Here a small example how you handle such a situation:

#if UNITY_ANDROID
	musicFolderPath = "/storage/emulated/0/Music";
#endif

#if UNITY_EDITOR
	musicFolderPath = File.ReadAllText (Application.streamingAssetsPath + "/folderconfig.txt");
#endif

- Change your input. The Sidebutton(click) can be used with MouseButton(0) and the Return button is mapped to “Escape”. Of course you can use more input, but you should ask yourself if this is really a good idea. We made the observation that a lot of people are already confused enough with VR such that a more complex input schemes can make a game quite inaccessible.
We used a double click as an input for some time, but it was not a good idea. Only using a normal click is better, since most people notice the click at some point, even if they do not read the tutorial messages!
Of course I also mention this, since your app should be able to use the back button in a consistent way, if you want to publish your app to the GearVR store.

Little Helper Android Debug Bridge

Since we needed to test our performance a lot, we needed to make a lot of builds and run them on a GearVR, since the results you get from it may not be the same as on the pc.
Using adb from the android development toolkit really helped to shorten the time between builds and of course it also helped to debug.
If you are not familiar with it, it is a program running in the shell which can install and debug your android programs on a connected android devise.
Our most used commands were:

-cd <Path>\Android\android-sdk\platform-tools
Getting to the correct folder in your shell to start using adb

-adb install -r <Path>\ShinyDev\Build\ShinySamsung.apk

Installing (and overwriting) Shiny to a connected mobile phone

-adb logcat *:S "Unity":D
Printing Unity Debug logs (and nothing else) from the android devise to the shell on the computer.

adb tcpip 5555

adb connect 192.168.X.XXX:5555

<Path> means the path to your folders and X.XXX stands for the IP of your phone in the wireless network.

Used to make a wireless connection to the android devise, this is especially useful when you have to put your android device in the GearVR.

Additional when you build your project make sure to select development build and autoconnect profiler (and set the profiler to deep profiling and adding the GPU window), to see the performance of your functions when running on the phone.

Optimization

Our performance before doing most optimizations.

You want your app to run with 75 frames/second, especially if you use the Oculus Rift pointer for controlling your game. Having lower frames can cause the game to feel imprecise, which is something we really wanted to avoid.
If you do not manage to get to 75 frames/second you will only get 37.5 frames which is often just not enough. (Well you can have some fast and some slow frames, but you want to have a consistent frame rate.)
I will talk now a lot about some basic optimization techniques, but in the end these little tricks are the reason we can run our game on GearVR with a good framerate.
 

OculusWaitForGPU and the 13.3 miliseconds

The GearVR will draw its frames once every 13.333 millisecond (which corresponds to 75 frames/second). If you miss this window, you will have to wait for the next frame meaning your frame will take 26.66 milliseconds (which coresponds to 37.5 frames/second). In the Unity profiler you will find under renderer the function“OculusWaitForGPU”. It seams that this process is making you slow, however this is mostly the waiting for the next frame. So all in all just try to be below 13.3 milliseconds per frame.

Memory and the Garbage Collector

The Garbage Collector hiding under the Update

When using the Unity Inspector and looking out for spikes in the framerate, you will often find the Garbage Collector, or sometimes you do not even find him (even with deep profiling it can occur, that you do not see him directly). Instead you see that some random process (allocating memory for the garbage collector) was slow.
This happens if the garbage collector starts his work during a process an, then the time he needs to do his work is added to the process which was disrupted.
This has two major consequences:
First be sure that the process you want to optimize is really slow (look at several frames/runs!) and was not just you looking at the garbage collector.
Second you do not want the garbage collector to be called often, since he will need 6-9 milliseconds to clean your memory and so you will most likely lose a frame each time he is called.
You may not be able to elude the garbage collector completely, but at least you can reduce the number of times he is called. In my opinion this is important, since every garbage collector call will lead to a spike in the time used for a frame, and such spikes is what causes framerate drops.

Calling the Garbage Collector manually

A transition in our game, a good time to call the garbage collector.

System.GC.collect lets the garbage collector run now. This can be a good idea to do when you have some points in your game, where it does not really matter if you have a slow frame.
In our game we can safely call the garbage collector, when changing a sequence or when having loaded the level. This may not make a big difference, but it can help a bit and can be done easily.
 

Reducing memory used:
 

Reducing the amount of memory used is not only good, to keep the garbage collector away, but it can also often speed up your game, since allocating memory can also be quite slow.
Of course it would always be best to program your game from the beginning in a way, such that it does not waste memory. However, often you want to have a working prototype first and or you are using some (not optimized) external tools. In such situations it may be a good idea to optimize the memory usage at a later point, as we did. Bellow are some easy ways we used to reduce our memory, if you want to know more you can find really nice guide here:

Functions calculating arrays:

When a function returns an array it will be copied and, therefore, a lot of extra memory is allocated and then given to the garbage collector.
An easy way to bypass this is to pass an array to the function and let it fill it instead.
This way the memory is not allocated twice and the garbage collector gets no extra work.
Farther below will be an example showing how you should calculate arrays in functions.

Reusing variables (especially Arrays):

If you need to calculate a certain value (or even an array of values) several times, it is better to instantiate your variable as a class variable and refill it every time.

So instead of allocating a new int array in every update, just create the array once, and fill the same old array again and again. If the array has a variable size, just keep an extra variable telling you how much is filled at a given time (and allocate it with the maximum needed space). Here is a combined example for the last two points.

Bad version:

using UnityEngine;
using System.Collections;

public class  ArrayExampleBad: MonoBehaviour {

	public int sequenceLength;
	public int sequenceLength2;
	public int strength;

	void Update(){
		int[] sequence1 = calculateSequence( sequenceLength);
		int[] sequence2 = calculateSequence( sequenceLength2);
		int minLength = (Mathf.Min(sequenceLength,sequenceLength2);
		int result = 0;
		for(int i=0; i<minLength; i++){
			result = result +  sequence1[i]+sequence2[i];
		}
	}

	int[] calculateSequence(int length){
		int [] temp = new int [length];
		for(int i=0;i<length;i++){
			temp[i] = length-i; // just a simple example!
		}
		return temp;
	}
}

Better Version:
 

using UnityEngine;
using System.Collections;

public class  ArrayExampleLessBad: MonoBehaviour {

	static inc MaxLength = 200; // depends on your max expected Length
	public int sequenceLength; //Now also used to count the length of the array
	public int sequenceLength2; //Now also used to count the length of the array
	public int strength;
	int[] sequence1 = new int[MaxLength];
	int[] sequence2 = new int[MaxLength];

	void Update(){
		calculateSequence(sequence1, sequenceLength);
		calculateSequence(sequence2, sequenceLength2);
		int minLength = (Mathf.Min(sequenceLength,sequenceLength2);
		int result = 0;
		for(int i=0; i<minLength; i++){
			result = result +  sequence1[i]+sequence2[i];
		}
	}

	void calculateSequence(int[] aSequence, int length){
		for(int i=0;i<length;i++){
			aSequence[i] = length-i; // just a simple example!
		}
	}
}

The second version is called less bad, since it could be optimized further, one examle would be to calculate the strength only when the sequenceLength changes, instead of calculating it in every Update. An example for something similar is given further down under Strings.
 

Using Arrays instead of Lists

When adding an element to a list, and then deleting it again, there is always some extra memory allocated and given free. If you use an array instead this is not the case and you can additionally use the two “tricks” mentioned above, to further minimize the amount of memory used.
Naturally there are some situations when having a List makes more sense, however, often you can use an array instead of a list if you use an additional variable to keep track of the number of used elements. Above this was done by “abusing” the two variables sequenceLength and sequenceLength2.

Strings and setters

Often you may want to use some strings for displaying information in your GUI. This makes sense and cannot really be evaded. Nevertheless you should try to not allocate too many strings. Especially concatenating strings and formatting strings will lead to new memory allocations and thus to some work for the garbage collector at some later time.
So instead of always concatenating two strings in every update, you could just make two strings in your GUI next to each other and change them individually.
Also only change the number of points/ life etc. when they have changed, and not just in every update. Further, you could even use a pre- calculated array of your possible strings, such that you can just link them rather than constructing them again and again.

Here is an example of only setting strings, when the points you want to display changed.

Instead of having the function:

public Text pointsTextField;
public int points;
Update(){
	pointsTextField.text = points + “ Points!”;
}

You could rather have:

public Text pointsOnly; // next to it a non changing string with “Points”.
private int points;
public int Points {
	get{
 		return points;
	}
	set {
	    points = value;
	    pointsOnly.text = value;
	}	
}

Naturally this makes only sense, when your points change less then once a frame (which they most likely should). 
The set function in a property can also call functions, this can be quite handy and is often better than just checking in the update for a change.

Using an Object pool for often created objects.

Using pooling we only need 2 shaders 2 meshes and 6 textures in this szene.

In a lot of games, you need to create the same objects again and again only to have them destroyed some seconds later.
Sometimes this objects are enemies, other times just the shots fired from your weapon.
In Shiny we have our nice little cubes, which often do not survive for too long.
If you have something similar, it might be a good idea, to not instantiate this objects every time and destroy them again. Instead you could use “pool” where you have a certain amount of these objects stored, and whenever you need one, just get one from the pool, and when they are destroyed return them to your pool again. So instead of using:
enemy=Instantiate(EnemyPrefab); and enemy.Destroy(); you could use
enemy= ObjectPool.GetEnemy(); and ObjectPool.Return(enemy);

The object pool itself can easily be made with an array of objects and a variable keeping track of how many objects you currently have stored. A really basic implementation could look like this:

ObjectPool{

	Enemie[] PooledEnemies;
	public Enemy enemyPrefab;
	private static int maxEnemies;
	private int count = maxEnemies;

	Start(){
		for(int i=1;i<100;i++){
			PooledEnemies[i] = Instantiate(enemyPrefab);
		}
	}

	Enemy GetEnemy() {
		count--;
		PooledEnemies[count].gameObject.SetActive(true);
		return PooledEnemies[count];
	}

	void ReturnEnemy(Enemy anEnemy){
		PooledEnemies[count] = anEnemy;
		PooledEnemies[count].SetActive(false);
		count++;
	}
}

Of course when having more complicated objects and when you have some initialization for them, this might become a bit more complex, nevertheless the idea is the same.

Pooling Meshes and Materials

Similar to the problem with the garbage collector, we had a problem with our camera renderer.
Since we instantiated new enemies with meshes and especially with materials (and with them textures) all the time, at some points the renderer was full and needed to clear the unused memory.
This caused also spikes and made the game less fluent.
Additional we had A LOT of draw calls, since with all the meshes unity had a problem with the dynamic batching (an option which can save a lot of performance!) which slowed down the GPU.
To solve this, we just used 1 Mesh for all enemies, which was set by the ObjectPool (after they were created), not by their creation itself. Before this optimization could often see BatchRenderer.Flush in our profiler taking its time, and afterwards this operation was hard to even find.
We used the same procedure for the materials (the only difference here is, that you need several materials for different colors). Using a single mesh and only 6 materials was another reason, why our object pool was so important in optimizing the game!
However, our object pool brought also a challenge, you will find it a bit further down, after an important note.
 

Making frames faster:

Having less memory allocations helped a bit to make our code faster, the object pool also helped a bit, but was not enough on its own. Even without the spikes from the garbage collector we did miss sometimes our target framerate.

After our optimizations we have almost no GC alloc and mostly fast frames.

OPTIMIZE THE IMPORTANT THINGS!

When you optimize your code It is important to find the parts in your code, which are not really efficient and try to increase their performance, so do not try to optimize functions only needing 0.3% of the time per frame, instead try to find the parts which are causing your game to run slowly and make them run faster.
I know it may be annoying to see some badly coded part of in your project, but unless you ARE SURE to be able to fix it in an instant, do not waste your time with code taking only a small amount of time to run. You most likely have more important parts which need your time. You can still write a list consisting the parts which you would like to clean up at a later time.

Moving Static Meshes or lots of Childs

Back to the Object pool and the meshes:
Since we just use geometric figures, we did not gave our enemies any rigid bodies, this however was a mistake, since unity thought this were static meshes and moving them was sometimes quite slow. This is something which was easy to fix, we just gave the objects a rigid body, (with no weight or gravity etc.)
Another white slow operation was moving and rotating objects with a lot of childs.
Because of our object pool it happened, that each of our enemies got about 5 child objects (with particles and other stuff which could be activated when needed).
As we noticed this, we stored this objects under a child in the object pool and only transferred them to our enemies when needed.

Functions being called a lot.

When you have functions, which take only a really small amount of time to run it can happen that the time they use is not displayed correctly.
Lets say you have a multiplication which takes only 0.001 milliseconds. Often it can be the case that this is rounded to 0.000 milliseconds. So even when this function is called 5 000 times it may be that the time displayed for the multiplications is only 0.02. However the “parent” function calling this function 5000 times may display high amounts of time used.
It is even worse, when you have a helper function “MyHelperFunction”, which is called 1000 of times but from different other functions, this makes it even harder to track the time needed to processing this function.

If you have a function which is often called, it may be worth optimize that function even if the displayed time for this function is almost 0. So whenever you have a function which is often called, try to make that function as fast as possible.
You can (for Debug purposes) even try to make the function slower (by adding something) or faster (by making it do nothing) in order to see, if it is worth spending time on optimizing this function.

We had a function calculating the square root in our code, and by letting it use a faster square root method (System.Math.Sqrt() instead of Mathf.Sqrt() ) we could fasten up our code quite a bit even though the displayed time for the square root was almost zero.

Converting Numbers (and other types).

Beware the above change was only faster, since we had the numbers from which we wanted to calculate the square root as doubles anyway. Converting from float to double is quite slow and we could safe some time by reducing the number of times we had to convert from double to float in our code.

Unfortunately we use two toolboxes one giving doubles, the other needing floats, such that we still need to do some conversion (and we did not have time to rewrite everything).
However, when you write code yourself, try to code in a way such that you do not need such conversions, since this leads to remarkable (and unneeded) time expenses.
 

The Update() function

Not every cube needs its own update function!
 

I know it is convenient to put a lot of things into the Update() function, but it also makes your code quite slow! A lot of things do not need to be calculated every frame. A great example is having the displayed string change when the points change (example above).
There are a lot of other different examples!
Do not let enemies look if they are hit during the update, let them do stuff, when they are hit.
Do not check if triggers are met, but instead call a function when they trigger etc.

Additional, if you do not need to do something during your updates, do not have an Update() function, it may not use a lot of performance, but since it is still called it may add up if you have a lot of objects.
If you need to do something after a given time, or only for some given time, it may be better to just have a Coroutine handle this task, instead of having an Update() function with some ifs waiting for the certain timespan.

Something different you could try is to make a helper class calling some “MyUpdate()” functions in other classes only for some specific times.
We have such a function and it helps to have when you need to run some functions only during certain time periods.
If you do not want to implement such a helper function you could instead use a counter for certain functions helping to calculate them only Xth frame, this may still be not optimal but at least a bit better.

Int count=0;

Update(){
	count = count +1 % X; // is 0 every Xth frame.
	if (count==0){
		DoCalculation();
	}
}

The nice thing about having such a counter is, that if you have a lot of objects (not being created at the same time), that they will (with high probabilities) be desynchronized, meaning they are not all called in the same frame and let you have less spikes.
When you have a centralized call every Xth update it is a bit more complicated to get the same result (but it is doable and most likely a bit more efficient).

Only Calculate what you need (and as precise as you need it)!

We are using a tool to analyze the music which is playing. The tool was quite slow on GearVR, so we had to optimize it. Fortunately we could change the source code.
The tool can analyze a lot of different things, but fact is, we do not need most of them during runtime!
When the core game runs, we only need the analysis of the beat (and volume).
So one of our biggest improvements was reached when we turned a lot of analysis off (by just commenting them out).
Additional the toolbox did Analyse Beats which take up to 8 seconds, but for our case, this is not really needed, it is enough when we only can detect beats of length at most 4 seconds, so we changed the detection range and gained again a lot of efficiency!
Further we do only really need to recognize beats, which (most) humans also can recognize.
Changing the beat finding algorithm slightly to only consider really clear beats (and chose the best from among them) improved the toolbox further since the number of possible beats to be analyzed was reduced by a big margin. It may be a bit less precise, but this is nor really remarkable during play.
It is often the case in games, that it is not important that what are 100% correct or 100% precise, as long as it seams to be correct for the human eye or in this case the human ear.

Test every “Optimization” !

I mention this, since sometimes you think “oh this will be much if I change X to Y”, but in the end it only makes it slower. At least this is the case for me, since I do not know exactly how fast each operation runs on unity or how the code is translated..
On the other hand, this also means, that you sometimes can make things more efficient, by doing stuff seemingly “silly”!
As our example above, I wouldn't have thought that the square root with the doubles would be faster! (I only tried it because I wanted less translations from double to float).
Of course such kind of changes make no sense to test in every function, but you can do it in functions which are called a lot or which are just slow for no clear reason.
Worse then just making the game slower is when changes break your game, but as you should know, you can always add some additional bugs to the code, even when “just” optimizing.

(We tried to interpolate some of the movements of our enemies on the spline, since the direct calculation in every frame was quite expensive, however, this was perceptible by the eye and just looked bad!)

Writing Custom Shaders

On the top you see our old background, on the bottom our new shader.

I do not mean that you should write all your shaders yourself, or try to optimize bloom Shaders etc. Nevertheless even if you are not that much into shaders, sometimes you can gain a lot by writing some simple custom shaders. We use several custom shaders, for different purposes and they brought the third biggest performance gain. (Right after the pooling our enemies including materials and the changes to our rhythm tool).
This page helped me a lot to understand the basic concept of shaders, and of course I also had a nice teammate explaining it to me even better;

Background Texture Shader

Before porting to GearVR we used several semitransparent layers with different shaders on them to create an appealing background. It looked nice, but it was not the fastest way of doing things.
Now we use a single NOT transparent material for our background.
We could make this change by using multiple textures in a single shader and calculating the overlay in the texture. Ir order to make the background moving we added (different) offsets to our textures, which are changed during runtime.
So more or less we calculate how a pixel would look if different layers were behind each other and paint our background with the corresponding color.
One can hardly see a difference in the visuals, but one can see a big difference, when it comes to performance.

Variations

Since we wrote a quite general shader, with some different parameters, we can use it to change our background on the fly, by simply changing textures and colors.
With this we can get different looks, depending on the music playing.
In the pictures below you can see different examples with the same color schemes. In game we use different color schemes for the different textures, however, it is easier to see the differences this way.
In VR they are even prettier and we get some nice “fake” 3D effects using them!

Two of our background varients using the same color scheme.
 

Rhythmic Cubes (Expansion Shader)

In our game the cubic enemies pulse on each beat meaning the change the size according to the intensity of the current beat.
In our old version we did this by changing the size of the cubes, but this was not really inefficient. Now we use a simple transparent vertex shader, which visually changes the size of the cubes, by increasing the distance between the center of the cube and the vertices.
The only fancy trick we needed is to have cube meshes, where the normals point in direction of the corners of the cube instead of in the direction of the faces. This is needed to ensure that the cube stays together when expanding in direction of the normals
(It looks nice though, when the faces go away from each other, however, this is not the look we wanted.)
Changing size in direction of the position of the vertices does not work as a simpler solution, if you want to allow dynamic batching. So this is why we used our small trick with the normals.
We can do this, since our material does ignore lighting, and having “wrong” normals does not harm.

Rainbow Shader

Since we were playing with shaders anyway, we also created a custom pulsating rainbow shader. The shader changes the size and color of our basic boss according to the beat.
We abused the fact that we know the position of our boss, and we can use the position of the center when calculating the colors. Each directional axis corresponds to one of the 3 color channels.
Additional to a simple increase in size we also added some small movement of the vertices in the shader (away from the player such that you cannot really see it, since it is canceld out by the increase in size) to change the colors slightly depending on beat.

A rainbow colored boss can bever be wrong;)

Further changes

Menu

We also wanted to improve our menus a bit in this version, since the old ones simply did not match the game. Our game is about colored cubes, the menu should somehow represent that.
Since our game is Psychedelic we first tried to create the menu to also represent this, however, it did not work out too well.
Even when, or maybe especially when, the game is crazy the menu needs to be easy to understand.
So after some more crazy experiments we came to a still cuby, but more relaxed version of our menu. It is still not final, but it is a big improvement over our “temporary” menu we had before.

Our difficulty menu over time.

Improved Point System

The game can be fun or relaxing, however it was never really challenging. To improve this we changed our point system, such that it is now more challenging to try to go for high scores.
Destroying cubes gives point according to your current multiplayer, missing enemy groups lets you lose part of your multiplayer, and in order to increase, or even keep the current modifier you need to constantly destroy whole enemie groups. This makes it more challenging since you cannot just randomly move around destroying everything and still get a good score.
With this more challenging system the bullet time feature, which we implemented some time ago is now actually needed rather than just a nice effect.

Small Changes

We also did a lot of other small tweaks, like changing color schemes, vary our standard music, fixing small bugs, adjust hitboxes, implementing a loading screen, changing particle forms etc. All small things which you would not really notice without having a comparison to the old version, but you will hopefully notice them as a whole.

The end

Definitively not the end of (the development) Shiny, but the end of article.

You can not only find us on our website but also at GDC this Wednesday.
Our talented Janina Woods  will be there, and would love to present you the game in VR.

I hope you had fun reading the article or learned some things. If you have questions or remarks, do not hesitate to write a comment.

Cheers
Marcel