The Power of Local Cubemaps at Unite APAC and the Taoyuan Effect

Introduction

Have you ever heard about the Taoyuan effect in reflections? Probably not, so I invite you to read this blog to learn about this effect and the application of local cubemaps in graphics.

Several blogs published in this community cover different uses of local cubemaps. The first was about reflections  followed by two more blogs that describe novel uses of local cubemaps to render shadows  and refractions  with great quality and performance.

These new techniques, completely developed by the ecosystem demo team in ARM’s Media Processing Group, are especially relevant to developing for mobile devices where runtime resources must be carefully balanced. They not offer only great performance but also high-quality rendering, which makes these techniques appealing to desktop computers and consoles’ developers as well. Fetching a texture from a static cubemap guarantees very precise and stable shadows, reflections and refractions, compared to the pixel instabilities we would get when rendering these effects at runtime with a moving camera using conventional techniques.

Thanks to these different uses of the local cubemap techniques, we can achieve very high quality graphics with existing hardware.

Unity Unite events in Asia

Recently Sylwester Bala and I attended Unity Unite events in Seoul, Beijing and Taipei where we presented the talk “Enhancing Your Unity Mobile Games”. I delivered the talk at Seoul, Nathan Li at Beijing and Sylwester at Taipei. In Seoul and Beijing there were joint talks with Carl Callewaert from Unity (see here).

In the talk we introduced the concept of local cubemap and how it can be used to render reflections, and we expanded this technique for rendering shadows in an innovative way. During the talk we used several short videos to illustrate the concept and the advantages of these techniques, which helped to deliver a clear and understandable message to the audience. A couple of videos were particularly useful to show how our new shadows technique can render dynamic and soft shadows.

In the final part of the talk Carl gave a live demo about some of the most important improvements in Unity 5: Global Illumination, Physically-Based Shading and Reflection Probes. Unity’s implementation of Reflection Probes is based on the local cubemap rendering technique. Now Unity developers have access to a simpler but powerful technique to add reflections in their games in a further optimized way, which is particularly relevant for mobile platforms. Carl also mentioned that Unity might consider implementing the new shadows technique in the engine as it is tightly close to the reflection probe feature.

In all cities, the talks were received well with great expectation and had a high number of attendees, reflecting the interest in these techniques. Carl showed a live Unity demo with a great example on how reflections based on local cubemaps work. His example clearly demonstrated the advantages of this technique and how easy it can be used in the Unity engine.

Reflections and shadows in games

During a long night walk in Seoul, Sylwester and I talked about how reflections and shadows can contribute to improving the visual experience of games. We walked along Yeongdong Avenue and there were high buildings full of neon and glass. Reflections and shadows were practically in every surface. Modern cities are full of polished, high-reflective glass and metal surfaces, virtually everywhere.

Therefore, when rendering this kind of environment in games, we need to consider the fact that the local cubemap technique might offer a very efficient way of rendering reflections and shadows in combination with other rendering techniques.

It is well known that reflections and shadows are key topics in any game. Without reflections and shadows any virtual world would look plain and unrealistic. Let’s take for example reflections. Reflections change every time the camera updates its position and orientation. We could prebake in a texture the reflections from the static environment but the final effect will be disappointing. An effective way of rendering high quality and optimized reflections is using local cubemaps. Using this technique we prebake the environment into static cubemaps and later at runtime we render the reflections by retrieving the texture from the cubemaps, using the local corrected view vector and then combining the contribution of local cubemaps for a certain position.

And what do you do with dynamic geometry? We obviously can’t prebake the reflections from dynamic geometry. In this case, we can render at runtime reflections/shadows from dynamic geometry using traditional techniques and combine them with reflections/shadows rendered using the local cubemap technique.

Unfortunately, the use of local cubemaps in reflections is not yet widely implemented, despite the fact that it has been available for 15 years. Now with the implementation of the Reflection Probe in Unity 5 the technique of reflections based on local cubemap is becoming available to more than the half of developers all over the world, which is pretty good. To provide support for our shadows technique based on local cubemaps in the Unity engine, it would be as simple as rendering the transparency of the environment into the alpha channel of the same cubemap used for reflections.

The Taoyuan effect

At the very end of our journey when heading to the flight gate at the Taiwan Taoyuan International Airport in Taipei, Sylwester and I were walking through a long corridor with a very polished and reflective floor. On one side the wall was projecting some message with Chinese symbols, which were perfectly reflected on the floor. The picture drew my attention and I pointed out to Sylwester that was a clear use case of reflections based on local cubemaps.

Nevertheless, Sylwester, who always pays extra attention to details, pointed out that further away from the wall reflections were more blurred. We looked at each other because it was clear that this effect can be implemented in reflections in the same way we implemented soft shadows.

In the shadows technique, we implemented the blurred effect by fetching the texture from an interpolated cubemap mipmap level. The magnitude passed to the fetching function is proportional to the distance from the fragment to the intersection point of the fragment-to-light vector with the scene-bounding box (a more detailed explanation of the shadows technique is in this blog).

As it happens with reflections based on local cubemap, our technique offers very clear advantages in terms of quality and resource saving when rendering shadows. Regardless of its limitations -mainly derived from its static nature- I would like to encourage developers to try the technique and explore it beyond the use case we presented in our talk. You never know what a technique can do for you until you start to use it and push it to new extremes. If you are skeptic about that then please continue reading.

The effect was caused by the fact that the Chinese symbols were carved in to the wall and the light source was behind in such a way that it was acting as a non-point light source that produced a soft pattern of light beams.

I decided to implement this effect in the same demo we used to show reflections and shadows: the chess room demo. The results are below. Fig. 3 shows the reflections on the chessboard using the standard technique of local cubemap. Fig. 4 shows a blurred reflection based on the distance from the fragment to the intersection point of the reflection vector with the scene-bounding box. The further the reflection is from the real object, the more blurred it is rendered.

This example shows how powerful and flexible the technique of local cubemaps is. It allows rendering of not only soft shadows but also the reflections coming from soft light beams.

Conclusion

What other new applications of local cubemaps will come? As local cubemaps are becoming more popular, developers will find new applications for this technique - it is just a matter of time. But one thing is certain; this simple technique has proved to be a very effective and powerful way to render reflections, shadows and refractions.