Realtime Shadows

Whether a game is 2D or 3D, shadows are invaluable for allowing players to judge relative distances between elements in the gameworld. While there are too many different methods to discuss here, one widely used in modern 3D games is shadow mapping. To create a sunlit scene, the game renders the environment from the perspective of the vector that defines the sun’s direction and stores the depth information in the Z-buffer. This creates a shadow map that defines everything that’s visible from the light source. When the player’s view is rendered, the game can compare each pixel’s position in 3D space to the map; areas in shadow can be rendered without the contribution from the light source. If the light moves, the shadow map must be updated, a process that creates moving realtime shadows. The most striking early use of the technique was in Splinter Cell, which incorporated shadows as a gameplay mechanic.

Ambient Occlusion

Realistic lighting and shadowing is a big challenge for 3D graphics. Consider a half-closed fist. If rendered with direct lighting, the palm would be lit as brightly as the back of the hand, and if shadowed, the palm would be in total shade. This does not happen in real life, because light bounces around, indirectly illuminating surfaces (see ‘global illumination’), and objects cast shadows on themselves. Ambient occlusion is a crude but effective way of simulating the way light bounces around an object. Though computationally expensive, it’s rarely computed in realtime – instead, the developer calculates it for each character, background or object as they’re created, to be combined with realtime lighting when rendered. The process shoots a spray of rays out from each polygon in a hemisphere aligned to the surface normal. If a ray does not intersect with another polygon in the mesh, the surface is made lighter, so that, in effect, polygons surrounded by many other polygons will receive less ambient light. It greatly enhanced the character models in 2004’s Half-Life 2, making them visually credible and lifelike to match the scripts they spoke and acted with.

Voxel Rendering

Voxel (or ‘volume-pixel’) systems represent scenes using a series of cubes. While a 2D pixel image is analogous to a series of coloured dots on virtual graph paper, a 3D voxel image is the same but extended into the third dimension, so that space is divided up into a grid of tiny cubes that can be transparent (air) or opaquely coloured (matter). Full volume rendering systems are incredibly computationally expensive and are currently the preserve of medical imaging and scientific visualisation, but a subset of the technology enjoyed brief popularity in gaming during the 1990s with PC helicopter sim Comanche Maximum Overkill. To reduce the load, Comanche used a 2D height field image to represent terrain. In a system similar to Doom’s ray-casting, Comanche traced rays for each column of screen pixels, and then computed where these rays intersect with a height field to render a vertical scan line of voxels.

Anti-Aliasing

While complex filtering can be considered to anti-alias textures, a different technique is required to remove ‘jaggies’ from the edges of polygons. The most common form is multi-sample anti-aliasing (MSAA) which is available in most modern GPUs and is done by taking ‘samples’ at slightly different positions inside each screen pixel (normally between two and 32 times) and averaging these to get a final pixel colour. As more samples are taken the fidelity of the result rises, but so does the computational cost. An ‘adaptive anti-aliasing’ technique can minimize sampling whilst maintaining quality by comparing samples as it makes them – if they differ greatly, sampling continues until the result becomes stable or a maximum number of samples is reached. Now a standard component of any 3D game for PC, many PC gamers’ first experience of anti-aliasing came when playing Tomb Raider on a 3DFX card.

Normal Mapping

While texture mapping can add detail to a surface, it can’t do everything to lend it a sense of materiality, such as the subtle variations in lighting that are formed by bumps and divots. Normal mapping addresses this while avoiding the need to give objects greater polygon counts by adding a texture to the surface that holds a ‘bump’ image. One colour channel in the texture indicates the horizontal surface normal perturbation (its deviation from the normal) and another channel holds the vertical perturbation. This texture is then mapped on to a surface, but the two-channel perturbation is used to adjust the surface normal in the Phong or other lighting calculation. In effect, it simulates the lighting of bumps on the surface. Normal mapping is an extension of the older and simpler bump mapping technique, which has also been widely used in games. It was first used to powerful (though many would say too powerful) effect in Doom 3, which boasted high detail despite the game’s relatively simplistic geometry.

Parallax Mapping

The development of surface rendering techniques has gone hand in hand with a steady increase in the power of the GPU. Parallax mapping is one of these, an extension of normal mapping in that it gives an added illusion of depth to a surface. By using a spare colour channel in a normal map to store the height of bumps on the surface, the game can create a parallax effect within the texture, which is calculated by displacing the coordinates at which the texture is read when rendering each pixel according to the view angle and height map. So, as the angle between the polygon and the view increases, the parallax displacement also increases. There are problems with this approach: notably, the effect breaks down at very oblique angles. There are solutions to this, but they’re expensive to compute. Some of the best examples of parallax mapping to date are the brilliantly chunky stone walls found in the cities of Oblivion.

Parallax Effect: The visual effect whereby close objects appear to move faster than those far away to a moving viewer. In 2D games, it is commonly simulated by scrolling layers of background graphics at different speeds.