The Golden Years of Graphics

If you ask me, the golden years of real-time graphics were the stretch where the original Xbox was winding down and the Xbox 360 was coming on the horizon. DirectX 8 and its Shader Model 1.x were handing the torch to DirectX 9, first with Shader Model 2.0 and then the big one, Shader Model 3.0. It wasn't just that the hardware got better. Developers had finally learned how to squeeze every last drop out of the hardware they already had. Some of the most impressive work of the whole era happened right at the end of that console generation.

Calling the engine inside Splinter Cell Chaos Theory "Unreal Engine 2" is technically correct, but if you look at what is packed inside, you may not even recognize it. Ubisoft took the same modified Unreal Engine 2.x that shipped in Splinter Cell 1 in 2002 and kept iterating on the engine for each game. Every Splinter Cell release was graphically impressive in its own right, but none of the graphics improvements were as big as Chaos Theory's. In 2005 it was arguably the best-looking game on the original Xbox, and on PC. It pushed every graphics boundary with the then cutting-edge DirectX 9, while still shining strong on older Shader Model 1.x hardware. Even in 2026 it still holds up remarkably well.

The Chaos Theory engine branch would later be forked again to power both versions of Double Agent, with one of them being the first "next-generation" console Splinter Cell.

Think about what that means for a moment. Splinter Cell was a series built on light and darkness: typically small rooms and indoor levels where most lights are dynamic and relevant to gameplay. Double Agent takes the same engine to light big outdoor levels in broad daylight. At the release of Double Agent, Unreal Engine 3 was the shiny new engine that developers wanted to build on. Ubisoft could have moved to Unreal Engine 3 at the time... yet they stuck with the then-aging Unreal Engine 2.

I thought it would be fun to explore this little engine that could power a decade of stealth action across GPU and console generations, with a specific emphasis on the Chaos Theory base engine and how it evolved into Double Agent and beyond.

Shadows are kind of the whole point

What makes both the game mechanics and the visuals look so good? Lighting! And you can't have a nice shadow without a good light. It's no coincidence that Ubisoft developed a franchise (mostly) centered around light and shadow mechanics at a time when real-time lighting at this scale was barely possible to do well.

Chaos Theory would represent a significant jump in lighting quality by implementing several new rendering techniques, including:

• Glow / Emissive bloom (All Splinter Cells had some version of emissive glow)
• Real-time specular highlights driven by the lighting system
• Per-pixel lighting
• HDR and tone mapping
• A new material shader system (replacing flat textures)

SCX Materials

The first two games' materials ran on Unreal Engine 2's fixed-function rendering, with no normal mapping or per-pixel specular lighting to speak of. Chaos Theory would be the first to make use of shader systems as we know them today, with real-time specular highlights, normal mapping and even parallax mapping on Shader Model 3.0 cards. Ubisoft called their evolution of the Unreal Engine 2 texture system the SCX Material, which contained a number of material types, including:

SCX_basic_material: This is the primary baseline material used by most surfaces. It contains the following settings:

• Base, Normal and SpecularMask textures;
• a HeightMap (that's the parallax mapping);
• an Environment cube map for baked reflections;
• Specular exponent, level and Fresnel controls;
• Diffuse and specular color (brightness, hue, saturation);
• An HDRMask for what blooms;
• A PunchThrough level for alpha cut-outs;
• A WaterLevel setting for wetness. SCX_glow_material: emissive surfaces that feed the bloom, with brightness, hue and saturation. SCX_reflection_material: mirror-like surfaces, with Dirt and CleanMask textures with a reflectivity level. (remember the cleaner dude with a mop?) ...and many more.

Even though these are real material shaders, Ubisoft's Unreal Editor 2.x still keeps them in the "Texture" browser:

The SCX Materials were used for both Splinter Cell Chaos Theory and Double Agent(s), and further evolved for Splinter Cell Conviction and Blacklist.

You might be surprised to know that Splinter Cell Double Agent doesn't have one shader system. It has two, co-existing together:

• SCX materials: Ubisoft's own system inherited from Chaos Theory. Most surfaces in the game use it and most of our SCX material types from Chaos Theory are present in Double Agent.
• Unreal Engine 3 materials: There's a material class literally called Unreal3Material, whose shaders were authored in an Unreal Engine 3 material editor, exported as shader code, and precompiled into the .ufx packages for Double Agent. They light a surface the UE3 way, with one pass per light, and they're used where SCX wasn't enough: skin with subsurface scattering, eyes and hair, refraction in ice, glass and water.

At a time when Unreal Engine 3 was considered the hottest engine around, Ubisoft didn't rebuild their game in Unreal Engine 3. Instead, they borrowed the pieces of Unreal Engine 3 they wanted and bolted them onto their heavily modified Unreal Engine 2 SCX build.

It's a notable decision their team had to make. I can only imagine how that conversation must have gone.

Dev A: "Everyone is talking about Unreal Engine 3, should we switch to that?"
Dev B: "We already added all the latest technology into our engine, way before Epic Games even released Unreal Engine 3. We already added real-time soft shadows, Havok physics engine, cutting edge sound dynamics with DARE audio, HDR, SCX custom material authoring, PATTERN visual scripting... What else do we need?"
Dev C: "UE3 material system kinda fire tho..."
Dev B: "Let's rip it out of Unreal Engine 3 and bolt it onto our SCX engine."

OK maybe it didn't quite go like that, but Ubisoft had every reason to NOT want to upgrade to Unreal Engine 3 as they had already developed their own version of the "new" graphics improvements that Unreal Engine 3 was only just then introducing.

It's worth highlighting some of those features 20 years later as a nod to that era:

Feature	Ubisoft's UE2 branch (shipped)	Stock Unreal Engine 2	First UE3 games (Nov 2006)
Lighting	Real-time, per-pixel dynamic lighting with normal maps (Chaos Theory, March 2005)	Per-vertex dynamic lighting on top of baked lightmaps; no normal mapping	Per-pixel dynamic lighting
Shadows	Real-time shadow maps from the actual lights since Splinter Cell 1 (2002) on NVIDIA/Xbox hardware; soft shadows in Chaos Theory (2005)	No real-time shadow maps. World shadows are baked into lightmaps; characters get a silhouette texture projected onto the ground from one fixed direction (or just a round blob). It ignores the actual lights in the room and isn't depth-tested, so it can smear onto walls and through floors	Dynamic shadows engine-wide
Materials	SCX material system: parameter-driven per-pixel materials (normal, specular, Fresnel, environment, parallax) and HLSL UE3 shaders (2006)	Fixed-function texture blending (Shader / Combiner / FinalBlend materials)	Node-based material editor compiling to HLSL
HDR & post	HDR, tone mapping & bloom (Chaos Theory, 2005); depth of field (Double Agent, 2006)	No HDR	HDR, bloom, depth of field
Physics	Havok	Karma	AGEIA PhysX (replacing Karma)
Scripting	PATTERN, Ubisoft's scripting system, since Splinter Cell 1	UnrealScript, Matinee, scripted sequences	Kismet visual scripting

In all things except faith, seeing is believing

Back to the lighting... so let's see how all of this came together.

"Glow" in the dark

Under the hood, glowing materials (lamps, screens, neon) get drawn twice: once into the frame like everything else, and once more into a separate, full-resolution glow layer that holds nothing but the glow (which then gets shrunk way down. More on that in a moment). Emissive materials have their own glow settings, so artists could decide exactly which things bloom and which don't.

To see it in action, I placed a simple tube light mesh with a SCX material containing a bloom layer on a plain concrete wall and captured it inside of Unreal Engine 2.x SCX:

Here's the same frame just before and after the glow is added:

Cue the aliasing!

The glow layers of Chaos Theory and Double Agent may be one of the first signs of the engine's age that you'll notice. The glowing materials are rendered at full resolution, but that layer is then shrunk to a fixed 256×256 square and smaller, and only those shrunken copies are stretched back over the frame. The result is a screen-door effect (aka aliasing) on every bloom and emissive object in the game. Here's how it works:

The glow as the stock 2006 renderer builds it in that same frame: the full-resolution glow layer, then 256, 128, 64 and 32.

The way the engine blurs the glow is, in my opinion, is one of the distinct artistic and signature looks of Splinter Cell. It's what makes the night-vision goggles, or a bright lamp in a dark hallway "bloom" in that signature Splinter Cell way.

On modern high-resolution displays, the game actually looks worse. Let's look at that first step again: 256×256 is a fixed square size, but modern displays are often widescreen (or ultrawide!), and the higher the screen resolution, the more pixelated a stretched 256×256 square looks. Even at just 1920×1080, every pixel of that glow buffer covers a cell 7.5 pixels wide and 4.2 pixels tall.

And because the grid belongs to the screen, not the world, the glow crawls when the camera moves. Here the camera pans slowly and you can see the emissive bloom popping in and out of the grid, creating an aliased bloom look typical of this era.

In 2005, on a CRT or an early LCD, this was almost unnoticeable!

In addition to the glow, the game also has an HDR bloom pass built on its own fixed-size squares, with the same kind of aliasing.

Lights ...Camera... SHADOWS!

The bloom and emissive paths of the game help contrast the dark unlit parts of the game. Shadows are not there just to look good, they are kind of the whole point. Chaos Theory's shadows are some of the cleverest engineering in the whole series.

A quick shadow map primer

Most real-time shadows of that era (and plenty today) use a shadow map. First, the game renders the scene from the light's point of view, but instead of colors it stores distance: how far the nearest object is in every direction. Then, when drawing what you actually see, it asks for every pixel: am I further from the light than what the light saw? If yes, something is blocking the light, and that pixel is in shadow.

A shadow map is effectively a picture, and pictures have pixels (texels). That yes/no question is evaluated once per screen pixel and every shadow-map texel shows up as a hard little stair step along the shadow's edge.

PCF, and why Ubisoft didn't use it

The standard for "smooth" shadows in 2004–2005 was PCF (percentage-closer filtering). Instead of asking the yes/no question once, you ask it several times at slightly different spots around the pixel and average the answers. Four "yes" out of four is full shadow, two out of four is half shadow, and the stair steps blur into a ramp. NVIDIA cards could even do a tiny 2×2 version in hardware almost for free.

But PCF has costs:

• It's expensive. Every sample is another shadow-map read, for every pixel, for every shadowed light. A 4×4 PCF is 16 reads where the hard test needs one, and a Shader Model 1.1 pixel shader can only read 4 textures in total.
• Cheap PCF still looks like a stair-step. A small kernel softens each stair step without removing it.

Illustrations from a simplified scene: the shadow of a pole with a ball on top, standing just off screen. Count the shadow-map reads per pixel: 1, 4, 16, and Chaos Theory's 1.

Chaos Theory's technical director, Dany Lepage, explained what they were after in a 2004 Beyond3D Q&A:

"Hard shadows (stencil or shadow maps) are very problematic from an artistic standpoint when they hard cut per pixel lit surfaces. The whole artistic concept of the renderer was to be contrasted but smooth, unlike SC1 which was contrasted and sharp/edgy."

And, about PCF:

"We do use DST but we don't use PCF! We were using PCF before but it wasn't giving very good results and it's now using point sampling."

When asked how cards without NVIDIA's shadow hardware would get the same quality, he said they had a Shader Model 1.1 algorithm that matched it on all hardware, and then:

"The details of this algorithm are confidential."

To my knowledge, no one has described how this confidential algorithm works.

So, 20 years later, let's unpack the mystery!

Step one: depth in disguise

The path that works on every card doesn't store the shadow map in a special depth texture at all. It encodes depth as color in an ordinary 16-bit (R5G6B5) color texture (1024×1024 by default, 2048×2048 at the max setting), written through a lookup ramp holding all 65,536 values. Plain color textures work on every DirectX 9 card, so this path doesn't depend on NVIDIA's special shadow hardware. In other words, it could work on ATI graphics cards as well. (On NVIDIA cards of the day, the game most likely used the hardware depth-texture path instead; that's the "DST" Lepage mentions. The color-encoded path is the works-everywhere fallback, and it's the one my capture took on a modern card.)

Step two: one sample, one decision

The shadow test itself is very simple: a tiny Shader Model 1.1 shader reads the shadow map once, with plain point sampling (no PCF, just as Lepage said), compares the depths, and makes a hard yes/no call. The result goes into the frame's alpha channel. On its own, that's the stair-stepped shadow in the top-left panel above.

Step three: the secret sauce

Instead of reading the shadow map many times per pixel like PCF, Chaos Theory reads it once, then softens the result afterwards, on the screen image itself, which costs next to nothing.

After the hard shadow is drawn, Chaos Theory takes a copy of the frame, color and shadow together, and shrinks it down in steps: full resolution, then 256, 128, 64, 32 and 16 pixels wide. Then it rebuilds it back up, one step at a time, but it only rebuilds the shadow. The color is just a guide that helps keep the shadow from smearing across object edges, and the visible scene is never replaced, so it stays sharp.

At every step of the rebuild, the softer, coarser shadow is blended back in, so the hard stair steps from step two melt into a smooth ramp. The whole thing is a handful of passes over small images per light, and one shadow-map read per pixel.

Contrasted, but smooth: exactly what Lepage was after, and built entirely from Shader Model 1.1 instructions with one shadow-map read per pixel. And because it costs so little, this shadow rendering pipeline could cross both the legacy and next-generation hardware, which meant it could run on PC from Shader Model 1.1 cards up.

As novel as this method is, it isn't perfect. The shrink-and-rebuild happens on a grid that's fixed to the screen, not to the world (sound familiar? Same as the emissive grid). So when the camera moves, a shadow that should sit perfectly still on a wall slides under that grid, and its soft edge changes shape a little from frame to frame: it shimmers.

Below is a controlled test. The camera slides sideways past the railing's shadow, with nothing else moving, and then every frame is shifted back in place so we can see what the shadow does when the camera is moving (without the camera actually moving). Drag the divider while it plays to see the shimmer in Chaos Theory "regular" SCX shadow mode or "HQ Shadow" mode:

The pattern repeats about every 7.5 pixels of camera movement, which matches 1920 ÷ 256: the size of the pyramid's first shrink step at 1080p. That's the screen grid giving itself away. In the regular mode the thin bar shadows pulse darker and lighter; the HQ mode isn't immune either.

At normal size and full speed you'd hardly notice it, especially on a CRT monitor, and on 2005 hardware that was a fantastic trade. This is also why, in 2026, the shadows in Chaos Theory actually look worse at higher resolutions than at lower ones: the filtering steps are a fixed size, which favors lower output resolutions (just like the emissive bloom layer).

Step four: the high-end mode

Lepage was open about where he wanted the PC version to go. In January 2004, a year before release and while the game was still called Splinter Cell - X, he wrote:

"At this point (things may change), I'm expecting that Splinter Cell - X will only support SM 1.1 and SM 3.0 when it comes out.

• Very significant performance improvement because of dynamic branching in the PS unit.
• Orthogonal FP16 operations
• Market (can't discuss that yet)"

On Shader Model 3.0 cards, Chaos Theory adds a dedicated soft shadows pass, and dynamic branching is what makes it fast. It finds only the small tiles of the screen that actually contain a shadow edge and skips everything else. Only those edge pixels get blurred; everything else is left exactly as the hard test drew it. The blur also shrinks the further the surface is from the camera, so the soft edge stays a consistent size on screen.

Why it was special

Lepage told the Globe and Mail that the game's lights were so expensive "that we needed some very efficient algorithms in that area." Instead of the PCF everyone else was reaching for, Ubisoft built:

• a real-time shadow map that ran on any DirectX 9 card;
• a soft edge from one shadow-map read per pixel, softened afterwards on the screen for almost nothing;
• and a high-end mode that only spent effort where you would notice.

The story of Chaos Theory shadows is where efficient novel algorithms meet artistic choice and hardware capability.

Some more examples from the SCX engine:

What Double Agent changed

Double Agent (Next Gen) rebuilt a lot of this. Remember all that effort to not use PCF? Well, the main lighting pass moved to PCF shadow filtering. 😅 The SCX shadow pass loses its regular and HQ shadow toggles in favor of two independent settings:

• Hardware or software shadow mapping. The hardware path ("HW Shadow Mapping") relies on the graphics card's own shadow-map hardware to bilinearly smooth the shadow edge. The software path does a multi-sample average instead.
• Shadow smoothing. An extra blur pass on top of either path.

SCX shadows also run at a lower shadow map resolution: 512×512 for most lights (1024×1024 for lights flagged high-res), versus Chaos Theory's 1024×1024 default and 2048x20248 for HQ soft shadows.

Quantity over quality

Ubisoft also added two new systems to the engine's lighting:

• Unreal Engine 3 screen-space shadow. Called the "Separate Shadow" or "Second Shadow", it's borrowed straight from Unreal Engine 3. (Notice a pattern here?)
• Baked global illumination. Up to this point, the Unreal Engine 2 lightmap baking was still (sparsely) used. Double Agent replaced that with a photon-mapped Global Illumination (GI) bake. Level designers place GI zones, GI-specific lights and GI "receiver" surfaces, and the editor fires photons around the level and bakes the bounced light into its own static GI lightmap.

Every lit pixel combines three shadow sources in one lookup: the real-time SCX shadow, a baked static shadow from the lightmap, and the UE3 screen space "Second Shadow".

Second Shadow

Here's how Unreal Engine 3 did it in 2006. Render a small shadow map fitted tightly around one object. Then, in screen space, take each pixel's depth, work out where that pixel sits in the world, project it into that little map, and write the answer into a buffer. Double Agent's Second Shadow follows the same recipe, except that it fits one map around all of a light's flagged casters. Its shaders even take the same kind of "screen to shadow" matrix as Unreal Engine 3's, and Unreal Engine 3's own shadow test shader (ShadowDepthTest.hlsl, still carrying Epic's 2003 copyright header) ships in the game's folder.

Why add a second real-time shadow system if SCX was so novel, you might ask? I think it comes down to two main points. First, the SCX shadow is literally called a "SpotLight" in the engine. It is designed to light and shadow a small, specific area, not a large one. Even if the spotlight were expanded to cover a wide range, performance and quality would suffer greatly. By borrowing Unreal Engine 3's "Second Shadow" screen-space technique, they can give each important object its own tightly fitted shadow map, so it stays sharp no matter how much ground the light covers. Second, it scales better: instead of redrawing every receiving surface for each light, it runs as full-screen passes over the scene's depth, with one fitted map per light covering all of its casters, which matters a lot on big outdoor daylight maps.

So what do Double Agent's real-time shadows actually look like? Here's the same scene again, but this time it's Double Agent's own renderer exactly as it shipped in 2006.

Depending on its settings, that renderer can draw this one shadow four different ways:

Top: the SCX shadow on the software path, with the soft-shadow filter off and on (pixel-identical, because the filter is broken). Bottom: the SCX shadow on the hardware path, and the "separate" shadow.

SCX shadows: software or hardware

They look quite different up close. The software path's edge is a staircase of three flat gray bands, one per step of its hand-made filter. The hardware path smears the same edge into one smooth ramp.

A light with separate shadows switched on takes the objects a level designer flags as separate-shadow casters out of its normal shadow map and renders just those objects into a second shadow map. Then, a full-screen pass takes the depth of every pixel on screen, projects it into that second map, and writes the result into spare channels of the screen's shadow buffer (one for characters, one for the world). The lighting shader takes whichever is darker, the normal shadow or the separate one.

The payoff is sharpness. The second map only has to cover a few objects instead of the light's whole cone, so the same number of pixels buys a far crisper shadow.

The main use case for these lights is seen in the daylight levels. The normal SCX shadow is one map stretched over the light's whole cone. That's fine for Chaos Theory's small dark rooms, but stretch it over a sun or a huge daytime light and every shadow-map pixel covers a big chunk of the world, so characters end up with soft, blocky shadows. A map fitted around just Sam, or just a tank, stays sharp however much ground the light covers.

It also fits Double Agent's lighting goals as a whole. The static world gets baked lighting and shadows from its lightmaps, and the objects that move (characters, vehicles, tanks) get sharp real-time shadows, with SCX spotlights for smaller areas.

Fun fact: why isn't every shadow in the game a separate shadow? Because it has a catch, at least with spot lights: anything moved into the separate path also drops out of that light's normal shadow pass as a receiver, so the light can no longer light it properly.

How does that compare with Chaos Theory?

Here's Chaos Theory's HQ soft shadow next to Double Agent as it shipped (the hardware path):

Up close, the difference is in the edge and the substantially lower shadow map resolution of Double Agent. Chaos Theory's edge melts out over a wide, even ramp. Double Agent's is a narrower blur.

If you are wondering why none of the four real-time techniques improved the look of Chaos Theory's design, you are in good company. We can only speculate about the exact reason, but my guess is a combination of a different studio inheriting a carefully tuned engine that it didn't build, and a race to ship a "next-gen" version on brand-new consoles, which may have meant biting off more than they could chew at the time. Also, "borrowing" entire segments of Unreal Engine 3 and bolting them onto your heavily modified older version of Unreal Engine is probably not the magic formula for optimization or quality control.

Global Illumination

The static Global Illumination system was an interesting addition: multi-bounce lighting, but baked in ahead of time. For a game that historically thrived on dynamic lighting, where anything can be shot out, baking in static lighting is a surprising choice. It was most likely aimed at the large outdoor levels, which needed some sense of depth without the performance budget for real-time ambient occlusion (screen-space ambient occlusion wouldn't debut until Crysis in 2007).

The images below show how the Double Agent editor builds its photon-mapped lightmaps.

Showing its age

Chaos Theory still looks remarkable for a 21-year-old game. Double Agent is a more complicated story. In most ways it's an obvious step back in real-time shadow quality.

Even so, there is so much more to show about this little engine that could. If this is something that interests you, let me know.