How Marvel's Spider-Man optimized its web-swings for PC gamers

PC versions of Sony-produced PlayStation games used to be the exclusive domain of copium-filled message board requests. Acclaimed, console-selling series like Uncharted, Horizon, or God of War on Steam? Keep dreaming!

A lot has changed in recent years, as modern Sony has committed to PC versions of its console games in a big way. Arguably the biggest sign of this initiative came from Sony's 2021 acquisition of Nixxes Software, a Dutch development studio that specializes in the arduous work of converting console-exclusive code to the wilder world of varied PC configurations.

At GDC 2023, members of Nixxes' graphics programming team explained how they completed their first project, a Windows port of the 2018 hit Marvel's Spider-Man. While the panel went into the nitty-gritty of its work converting the game to DirectX 12 and Direct3D 12, it also confirmed that Nixxes' prowess didn't lead to instant porting success.

"Look, a triangle!"

Nixxes programmer Michiel Roza described the first steps his team takes on any port, in line with its seven years of porting console-focused code to the Direct3D 12 API. After playing the game in question and examining its raw code, the team differentiates what it sees and parses it in three chunks: platform-agnostic code; platform-specific code; and code that appears to be cross-platform-capable but needs tuning. He mentioned a few "crude" tools that Nixxes could begin porting to accelerate various calls, which was no insult for what the console version's developers at Insomniac Games had left them; everything else had to be made from scratch.

When pressed on how long it took to get D312 graphical calls running on PC, fellow Nixxes programmer Rebecca Fernandez O'Shea described the team's process of stopping every function in the game's codebase, then cobbling together and running separate functions one-by-one until they finally saw an image appear on a computer monitor. O'Shea estimated that this process took six weeks, and Roza recalled his delight at what he saw appear: "Look, a triangle!"

If that seems like a long time to generate a single triangle, the Nixxes programmers hint at how much more arduous the process went in 2016, when it began its first port of a game to Direct3D 12. When pressed about their earliest D312 efforts during a Q&A portion, Roza offered a one-word answer: "Badly."

Winning the battle against #stutterstruggle

Roza and O'Shea's dive into Spider-Man on PC focused on five graphics-specific topics. The first focused on the highly technical phrase "pipeline state objects" (PSOs), but the Nixxes duo got the crowd's attention with an arguably more popular phrase on their GDC projection screen: #stutterstruggle.

This hashtag, as popularized by Digital Foundry PC gaming analyst Alex Battaglia, refers to the deluge of modern PC games that introduce frequent, microsecond-long stutters in the middle of gameplay, typically when a shader, object, or animation is loaded for the first time on a PC. This has been particularly endemic in Unreal Engine 4 games on PC, but Nixxes had a leg up by starting from scratch with Insomniac's codebase.

Roza detailed a number of options they could have implemented to mitigate such stutters, and one of these was a lengthy pre-load sequence that lasts between 5-10 minutes before customers can play their new game (as seen in the PC port of another Sony game, Horizon: Zero Dawn). This countered Nixxes' philosophy of getting players into new games as quickly as possible, Roza said.

Instead, the team went down a far more intensive road: asking its team of quality assurance (QA) testers to document every single PSO that the game calls for, a task that proved all the more time-intensive when the whole list had to be gathered anew thanks to major code updates like a "fundamental shader change." With this full list in hand, marked by memory-friendlier hash values, Nixxes' programmers built functions that cached crucial PSOs on the fly during the brief loading screens that already occurred when players switched from indoor to outdoor to cut scene moments—and assigned these caching functions to specific CPU threads in order to conduct them seamlessly.

In the rare event of a PSO not being generated quickly enough, Nixxes implemented a gray-box backup mesh for major gameplay elements, which is nearly impossible to see in normal gameplay but can be glimpsed in millisecond-long flashes if you force extremely high settings on a weak PC. A brief animation of the team's backup mesh at GDC looked like something out of a prototype N64 game.

More meshes, more rays

Roza confirmed that Nixxes built an entire system to accommodate players who ignore Spider-Man's recommendations for graphics cards (GPUs)—specifically, anyone who plays the game with settings that exceed the amount of video memory (VRAM) available in their GPU. Though Windows includes a memory management system that can shift memory-related calls from VRAM to general-purpose system RAM, this is unoptimized by default, and video games need quicker memory for specific tasks.

Nixxes' solution was to create its own non-unified memory management system, which ranked and tracked a hierarchy of calls' importance in the code, then measured them by "distance to 512 MB" in order to prioritize which large chunks of memory would make the most sense in a shift from VRAM to system RAM. So if you've ever decided to push your Spider-Man gameplay on a GPU that wasn't quite up to the task, you can thank Nixxes for implementing an invisible reduced-stutter option.

A detailed explanation of the game's ray tracing system, as ported from the game's PlayStation 5 version, confirmed that this toolset was largely built with platform-agnostic tools—arguably because there's little reason for a ray tracing pipeline to use wildly different methods while the technology is so young. Still, shifting Insomniac's code to PC required a few tweaks, including a new version of the game's "skybox" system and a fix to a bottleneck in how the PS5 version read from "UPLOAD" memory.

The biggest ray tracing-specific task came from efforts to unlock more realistic and detailed reflections on PC, since that platform features more powerful RT-specific graphics cards than found on modern consoles. When Nixxes built a "PS5-equivalent" RT mode, it relied on "impostor meshes" for its reflections, which included simpler versions of buildings, characters, and other objects. In order to max out PC GPUs, Nixxes had to build an entirely new RT mesh system from scratch, so that reflections would look far more detailed.

This, too, required heavy lifting from Nixxes' QA team, as every object in the game had to be flagged as part of this revamped, higher-detail mesh in case it incorrectly appeared in an RT reflection model. These included UI elements, misplaced shadows, and special effects lighting codes.

As the Spider-Man PC port neared completion, Nixxes was gifted Insomniac's newest RT-specific optimization: a "checkerboard" resolution option to upsample the number of pixels generated in an RT pass (as developed for Ratchet & Clank: Rift Apart on PS5). This led to better-looking RT imagery for any compatible PC, though Nixxes also implemented a demanding full-pixel option for higher-end PC players. "It's four times the rays per frame," O'Shea said. "Because that's cool." (The above image shows a zoom into the PC version's checkerboarded reflection default on the left and the fuller pixel reflection on the right.)

"BIG OOPSIE HERE"

O'Shea led a deep dive into Nixxes' error code systems, which the team used to generate more verbose troubleshooting language than is traditionally available on PCs. "PlayStation 5 has a really awesome, in-built tool for GPU capture in the event of a crash," O'Shea said. "We can see what happened in the frame directly before a crash. There's no such tool on PC."

Without such a handy built-in debugging option for PC, Nixxes relied on a pair of troubleshooting tools, Microsoft's Device Removed Extended Data (DRED) and Nvidia's Nsight Aftermath. These are only built into the game's internal, debug versions for QA testers to track. Both tools sap PC performance ever so slightly, O'Shea says, and that's a no-no for customers. One verbose print-out of a typical Marvel's Spider-Man crash report, which helped Nixxes hone in on a pain point, included an all-caps declaration: "BIG OOPSIE HERE." O'Shea called this "our very professional error message."

The panel concluded with the game's support for arbitrary monitor resolutions beyond the console version's 16:9 default, a feature that Nixxes has added to many of its earlier ports from consoles to PCs. In addition to popular ultra-wide monitors, PC players may want to arbitrarily resize the game's window, or even run a game in portrait mode. O'Shea confirmed that the team settled on hard limits for how arbitrarily a player could adjust the screen ratio. Once the ratio reaches 32:9, any wider screens from there would include a blurred indicator of colors and possible objects in the periphery; beyond that, anything wider than 41:9 receives black bars.

Nixxes included a number of examples of the original 16:9 game exhibiting bugs and glitches when stretched to a wider resolution. In some cases, like the above power-selection menu, UI elements either didn't cover the whole screen or looked stretched. In others, like the below dedication to Spider-Man co-creator Stan Lee, there was no way to alter the assets appropriately, so selective black bars were implemented.

In all of these cases, Nixxes internally created or modified existing assets to render at arbitrary resolutions. One cool example is an ultra-wide view of a first-person mode where Spider-Man's drone crawls through a tunnel. Insomniac had only rendered a tiny pair of robo-legs to appear in the players' periphery, which looked silly in ultra-wide ratios (see below for the unfixed version). As eventually updated by Nixxes, those legs now extend much further and look more menacing.