Ray Tracing in Games: How does it work?

Ray tracing technique was implemented back in 1980 when an American electrical engineer J. Turner Whitted described it as a novel method to compute 2D images of 3D objects.

In the paper “An improved illumination model for shaded display”, published in Communications of the ACM magazine, Whitted explained the concept of ray-tracing along with some other things like anti-aliasing that we still use up to this day.

For those looking for a quick glance, here are some key points about ray tracing:

• It is a process of tracking the path of light and observing how it interacts with transparent, translucent, and opaque objects.
• It makes the rendered objects, reflections, and shadows look lifelike.
• It is very complicated to carry out, and this causes a drop in FPS in games.
• GPUs that support it cost a ton. Nvidia’s new RTX 3000 series GPUs, AMD’s RX 6000 series GPUs along with Sony’s PS5 and Xbox Series X will bring some game-changing improvements to the technology. 

Before we get into the discussion, watch this video by Nvidia, which shows off a ray-traced game demo running on the latest Nvidia Geforce RTX 3000 GPU:

Here’s another example of the same by AMD using their FidelityFX Ray Tracing on Radeon RX 6000 GPUs:

    Fundamentals of Real-Time Ray Tracing

To appreciate ray tracing, we have to first take a look at the history of computer graphics. We are going to come across some terms that have been in use since the early 80s. It is necessary to understand how it all began.

What is Ray Casting?

Mathematically, a ray is the portion of a line that originates from a fixed point and extends indefinitely in a particular direction. A ray always travels in a straight line in a medium until it hits the boundary of the medium that it is traveling in. 

Once it hits that boundary, it can either get reflected, refracted, absorbed, or undergo all three operations partially.

Ray casting is a computational technique in which a ray coming from an object is made to fall on the observer. It is the most basic form of ray tracing and was used in the very first 3D games ever made. 

Games like Wolfenstein 3D was one of the first ones to use the ray casting technique. Wolfenstein 3D was released for DOS in 1992. I am sure a lot of the readers would recall playing this game in their childhood.

Rasterisation and Ray Tracing

Rasterisation is a prevalent method of rendering 3D objects. In fact, this is the method that is in use in computer games for decades now. 

In rasterization, a 3D object is first made up of millions of triangles. These triangles are then used to determine the shape of the object. The shape is texturised and presented to the user.

The computer had to find out exactly which pixels do the objects cover and make an image out of it. This process is repeated for 30-60 times a second when you play video games on your PC. 

The reason why this method is widely used in rendering 3D objects is because of how fast it is. The hardware that we have today is capable of performing the process of rasterization in real-time. Rasterisation can achieve a perfect balance between rendering quality and rendering speed. 

For a very long long time, rasterization has served us well. But this process of converting a 3D object to a 2D image and estimating its textures has its drawbacks. The picture quality is not life-like.

The reason is that during texturization, the computer has a hard time figuring out where to fill in the light. It gets impossible for the software to depict the way light interacts with the 3D objects; how it hits the surface and bounces back.

Ray tracing, on the other hand, is done by actually tracking the path of the light source’s rays as they hit the objects and illuminate the environment. Only the rays that are going to reach the observer are traced to reduce computational complexity. Here is a figure which shows this concept.

The one interesting thing that can be noted about ray tracing is that it takes into account the surrounding of the scene too. It is not necessary that only the objects present in the scene should have an effect on the final image.

A lot of times, objects that are present outside the scene like the sun, clouds, and the environment can have reflections in the scene, which would be not visible if ray-tracing is not performed.

Ray tracing has been used in high budget 3D movies for a while now. Popular movies like ‘Cars’ and ’The Toy Story’ by Disney Pixar Animation Studios also made use of it. So naturally, the question arises, if ray tracing is really that great, why isn’t it being used in every game?

The answer is: it’s very costly. Ray tracing is something that is really computationally intensive. It is still a lot simple to use it in movies as they are pre-rendered. There is no time constraint for rendering every frame in the movies.

Rendering a movie is a one time process, and it can take a number of hours or days to render the complete scene. It is done on a really fast computer or a server farm, and the result is a great-looking movie that can be viewed on any device. Gaming, however, requires you to render 30, 60, 144, or even 240 frames per second. 

This would bring even the most capable GPUs to their knees. There is no way real-time RT could have been performed on a consumer graphic card until Nvidia unveiled the Nvidia Geforce RTX GPUs lineup in 2018. The new high-end GPUs made it possible for gamers around the world to experience real-time ray tracing from the comfort of their homes.

This didn’t end the era of rasterization as the technique still remains widely used in all the new gamers. In fact, both rasterization and ray tracing can be used to make the most of the available hardware. 

Here is an image which demonstrates the power of Ray tracing: Here is another image which is generated by using both the rasterization and RT: 

    Games that support Real-time Ray Tracing

Ray tracing has been around for around 2 years now. The list of games that support ray tracing is still not very huge but there are some games that look absolutely gorgeous. Here is a list of some games that support it and you can play them right now:

• Control
• Call of Duty: Modern Warfare
• Shadow of the Tomb Raider
• Battlefield V
• Wolfenstein Youngblood 
• Watch Dogs: Legion
• Fortnite
• Quake II
• Godfall
• Call of Duty: Cold War
• Minecraft

Upcoming PC games that support Ray Tracing

If you are among some of the lucky ones who have gotten their hands on Nvidia’s latest RTX 3000 series GPUs or AMD’s RX 6000 Series GPUs, then you should be looking forward to playing these upcoming games to make the most of out of your GPU:

• Cyberpunk 2077
• Far Cry 6

Ray tracing on consoles

Both the PlayStation 5 and the Xbox Series X use AMD’s GPUs based on RDNA 2 architecture. These will be the first AMD GPUs to support ray tracing. A lot of new games have been announced for the consoles that support it.

Here is a list of PS5 titles that support ray tracing:

• Demon’s Souls
• Watch Dogs: Legion
• Gran Turismo 7
• Spider-Man: Miles Morales

And here are the games that are lined up for the Xbox Series X:

• Forza Motorsport 8
• Watch Dogs: Legion
• Halo Infinite 

    Is it worth it?

The ray tracing has been exclusive to Nvidia’s Geforce RTX GPUs for quite a while. Not even the Geforce GTX line up supported ray tracing. It was impossible to imagine playing games on 4K resolution, 60 FPS with RTX on. Up to this point, buying an Nvidia RTX GPU didn’t make sense unless you had a lot of extra cash to spare. 

The new RTX 3000 GPUs however, changed the game. The new Nvidia Geforce RTX 3070 performs better than the RTX 2080 Ti and costs less than half it’s price.

Not to forget, AMD’s new RX 6000 series of GPUs also support DXR (DirectX Ray Tracing).

Here is a table showing how the prices of GPUs that support Tay Tracing.

With the new ray-tracing GPUs getting more affordable with each passing generation, and a lot of new game developers implementing the latest technology in their games, we can expect ray tracing to make its way to a lot of entry-level and mid-range PC builds soon.