Introduction

Ray tracing has seen an immense growth of interest with the release of NVIDIA’s RTX video cards. We are already seeing these features in games today. At the time of writing this blog, Shadow of the Tomb Raider, Battlefield V, Metro Exodus, and Quake already use this technology. And during E3, even more games have been announced to include ray tracing. Just imagine the real-time correct reflections in Keanu Reeves’ eyes during a burning city scene! Or realistic indirect light, without placing extra light sources in game scenes.

Before the release of the RTX cards, ray tracing was mainly used in architecture and the creation of animated movies. Mainly due to the fact, that these industries are able to afford the time before a ray tracer (or path tracer) has rendered a frame. The game industry doesn’t have that luxury since games we run today have to meet our standard of 60 frames per second. To meet this requirement we currently use rasterization techniques to perform real-time rendering, mainly because hardware manufacturers have been creating specialized processors, called Graphics Processing Units (GPUs), to handle rendering using these methods.

In this blog, I will introduce you to a project I’m working on during my master studies, that tries to use only ray tracing to create the final image, presented by the renderer. This blog post is a very shortened version of my thesis. If you are interested in either the thesis or source code, I’d like to refer you to my site.

The Denoised Ray Tracer

With ray tracing, there has always been a trade-off between quality and performance. If you want an excellent ray-traced image, a lot of rays need to be processed by the computer, heavily impacting the performance of the project. If the developer aims for better performance, the ray count has to be as low as possible, resulting in a noisy image (Figure 1).

Figure 1 Image shows the noisy result from using only 1 sample per pixel for the final render.

This noisy image can be corrected through the use of denoising software. However, there are downsides to this approach. One of the downsides is that the quality of fine-grained textures can be lost in the denoising process. This problem has been resolved in OptiX 6.0, a denoiser made by NVIDIA, based on the paper of Chaitanya et al. “Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder” [Chaitanya et al. 2017]. The way OptiX handles this is by using an extra data buffer holding albedo data. The effect can be seen when comparing Figure 1 and Figure 2. Even through very obvious noise, the denoiser knows that there is a cobblestone texture assigned to those pixels, thanks to the albedo buffer that was specified.

Figure 2 Image shows the denoised result from using only 1 sample per pixel for the final render.

Results  

Performance Results

Comparing the differences in performance between the denoised and non-denoised raytraced techniques in Figure 3 shows a massive loss in performance to do the denoising. Figure 4 shows that the denoising takes 47% of the total render time, on a resolution of 1280 by 720 pixels. These results might suggest that a full ray tracing approach that is also denoised is not possible to run at real-time, however, ways of speeding up the process exist, but they were beyond the scope of this project.

Figure 3 Bar chart visualizing the time per frame in milliseconds, per resolution. (On RTX 2080ti w/ 11GBs of memory)

Figure 4 Image shows how much time the main steps take to render and denoise an image every frame. (On RTX 2080ti w/ 11GBs of memory)

Render Quality Results

The quality impact of a denoiser is clearly seen in Figure 5. The image made from a single sample per pixel (SPP) shows a lot of noise. This is expected since a random approach is used to achieve global illumination and soft shadows. The noise from a 1SPP image can be removed completely, resulting in a “clean” render. When comparing the 1SPP denoised images with sampled images there is a visible difference in the details of the render. The Rungholt (Minecraft) scene in Figure 5 for instance, clearly shows that a lot of the details in the leaves are lost.

Figure 5 Images from Crytek’s Sponza and Rungholt visualizing the difference in the number of samples taken per pixel, and compared to its denoised equivalent.

Future Directions

The main challenge of this project was maintaining a frame rate that could be used for games, preferably being 60 frames per second. This was possible using a limited amount of rays by lowering the resolution. To use raytracing in games, a developer or future researcher could focus more on the use of hybrid rendering techniques. This technique allows the performance strength of rasterization, and the realism increasing features a ray tracing approach can deliver. Hybrid rendering is being explained in-depth in an NVIDIA blog post [NvidiaDev 2018]. The blog post assumes the reader has a basic understanding of DirectX Raytracing, which the thesis I wrote on this research can establish.

In that blog post, Nvidia also mentions various denoisers for different raytracing tasks. The denoisers will be added to the Nvidia GameWorks library [NvidiaGW 2019] that, when writing this blog, is still in early access. Using different denoisers for specific tasks such as soft shadowing, glossy reflection, and ambient occlusion could increase the quality of the denoisers output because they are trained to perform a very specific task of the denoising process.

I hope that with this blog post I was able to hype some people up, for what the future in game graphics will bring.

 Note that these graphs and discussions are a very simplified version of data and discussion in my thesis. If you are interested in a more in-depth analysis and discussion I highly advise you to read it. The thesis can be downloaded from my site.

References

Alla Chaitanya, C. R., Kaplanyan, A. S., Schied, C., Salvi, M., Lefohn, A., Nowrouzezahrai, D., Aila, T. (2017). “Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder”. In: ACM Transactions on Graphics 36.4, pp. 98.1 – 98.12.

NvidIA Developer (2018). “Effectively Integrating RTX Ray Tracing into a Real-Time Rendering Engine” https://devblogs.nvidia.com/effectively-integrating-rtx-ray-tracing-real-time-rendering-engine/

Nvidia GameWorks (2019). “GameWorks Ray Tracing” https://developer.nvidia.com/gameworks-ray-tracing