64-bit Game Development: Immediate Changes Ahead

Author: by Alex Pournelle

While the Game Developers Conference is the big yearly show for game developers, Microsoft's Windows Hardware Engineering Conference (WinHEC) is a key conference for PC hardware companies, and is where Microsoft releases many technical details of its current and future software interfaces. WinHEC 2004's main topics were, unsurprisingly, Longhorn (the codename for Microsoft's next wave of platform and software releases, due in 2006) and 64-bit technology. In this article, I'll discuss some of the effects the 64-bit PC revolution on game developers--changes that will be felt well before Longhorn ships. I believe these effects are significant, for everyone from small developers through tool/middleware companies to the game mega-corporations.

I'll focus on AMD Athlon64 and Opteron parts, plus Intel's upcoming EM64T technology (short for Extended Memory 64 Technology). According to the most credible reports, EM64T is completely compatible with AMD's 32/64-bit chips. Intel's only apparent incompatibility, DEP (short for "Data Execution Prevention", more on that later), has apparently been added in recently; doing so may delay the release of "64-bit ready" Pentium 4 chips. Meanwhile, AMD's architecture has been licensed by Transmeta, and Via might follow suit, so I'll refer to all of them as "x86-64" rather than the more fraught (but equally apropos) "AMD-compatible". (I'm purposefully ignoring the Itanium, which is neither AMD-compatible nor a factor for gamers.)

So, the big two, Intel and AMD, are headed toward a fully 64-bit future with complete 32-bit backward compatibility. At WinHEC 2004, Bill Gates estimated that 64-bit processor shipments would out ship 32-bit ones by late 2005; this will largely depend on how quickly Intel can ramp up their 64-bit compatible production. Whether or not AMD keeps its current lead over Intel in CPU shipments, 64-bit computing is shipping and here to stay, even though the 64-bit version of Windows is still in beta.

Intel System Architectural Changes on the Roadmap

Intel, long the beneficiary of Moore's Law, dropped a major bombshell on the last day of WinHEC: The next-generation Pentium 4 chips, codenamed "Tejas" (workstation) and "Jayhawk" (server) were being dropped. The days of increasing clock speed as the main method for increasing processing power are apparently over. Consider for a moment that the business-as-usual product roadmap for Prescott would mean shipping 4 GHz parts today. Instead, we're two speed grades below that, the Pentium 4 Extreme Edition (a Pentium 4 with the cache size of the Xeon and a few other features) notwithstanding.

Intel has disbanded the teams that were working on those parts, at the cost of over a billion dollars in development. It appears the Pentium M (currently sold as a laptop part) will be the future main product, though it's not clear how soon they'll be able to add EM64T to it (credible estimates predict in the summer of 2005).

More immediately relevant is that Intel's future is apparently multi-core. Intel has been toying with two-CPUs-on-a-chip for at least 15 years; that research is now being put on the fast track. Intel has put dual-core products on the roadmap, to ship late 2005 at the earliest. The Opteron, with its HyperTransport link, is already shipping in dual (discrete) CPU configurations; AMD will follow suit with dual-core one-chip designs if they see a clear advantage.

Intelligent support for threading is, of course, a change that benefits most speed-sensitive Windows software, whether running on HyperThreaded Pentium 4s, dual Opterons, or future dual-Pentium M cores. The standard Windows 2000 and XP workstation licenses support one or two "real" CPUs (so a HyperThreaded Pentium 4 counts as one), and there are plenty of hot dual-Opteron game systems being sold. Recommendation: Threading should be a standard part of your development cycle if it hasn't already been. CPU counts are going to increase; I expect four-way Opteron workstations (not just servers) to ship by the end of the year, assuming Microsoft makes a 4-CPU license for XP Workstation available. (Right now, only Windows Server supports 4 or more processors.)

32-bit Windows Changes for Windows XP SP2

Even before Longhorn, there are serious changes developers must consider--for both 32-bit Windows XP and the upcoming 64-bit variant. Some of these changes affect all versions of Windows, regardless of processor type, while some (the Data Execution Prevention and large-memory features described below) only work on x86-64 systems running Windows XP Service Pack 2 (SP2). SP2 uses some 64-bit features in 32-bit mode.

Self-Modifying code will be seen as harmful. The "Data Execution Prevention" (DEP) flag, also known as "NX" (No Execute) by AMD may cause some programs grief. SP2 will have this flag enabled by default, and x86-64 systems will throw an exception if a program attempts to overwrite code with data, causing the program to terminate.

The DEP control panel is grayed out and has no effect on existing 32-bit x86 systems. But on x86-64 systems, if an application attempts to modify system code, Windows will put up an error message and force a reboot. DEP is one of steps Microsoft is taking to combat viruses and other malicious software in SP2, even before the 64-bit version of Windows for AMD systems ships. DEP should prevent many of the buffer-overrun exploits which have enabled everything from Sobig to Sasser to make major headlines, and will work even when users haven't kept their Windows patches up to date.

Self-modifying code (code that re-writes itself at run-time) has been used to boost the performance of videogames as well as to enhance code security and copy protection, but since it is nearly impossible to tell the difference between a program doing this for good reasons (as in a videogame) and nefarious reasons (such as, say, a virus or worm); if any of your currently supported products blend data and code, you will have to either change them or include instructions for gamers to open the Security control panel, select the DEP tab, and add the game to the list of applications which need to run with DEP disabled. You should also expect support calls from people installing SP2 and running existing games. As of this writing, SP2 is still in Release Candidate 1; expect RC2 by mid-June, with a final release by mid-summer. Recommendation: If you have any self-modifying code left in your development system, rewrite it now.

Support for RAM up to 4GB. Windows 2000 and XP allow applications to address up to 2GB of RAM, and 3GB if the rules for LARGE_MEMORY_AWARE applications are followed. As we reported on Byte.com in the April 12, 2004 Chaos Manor column (paid subscription required), XP can actually address considerably more, but that's a story for another day. Windows XP SP2 will allow applications to address up to 4GB on x86-64 systems. While this limitation is unlikely to affect most game developers today, you may want to start building in support for the larger memory capacity, so that when your high-end players are ready for the larger worlds and other features this will support, so are your games.

Browser changes. Another important change in SP2: Internet Explorer comes with a popup blocker enabled by default, so online games and browser-based features, such as online registration, will need to be modified to work without pop-ups. SP2 also changes how alerts for installations of ActiveX controls are handled, so test anything Internet Explorer-related under SP2 carefully.

Patch and update management. In XP Service Pack 1 and Windows 2000 SP3, the Windows Automatic Update feature (System control panel, Automatic Update tab) will, when enabled, cause your system to "trickle" (that's Microsoft's term for it) updates to your system. Once they're downloaded, Windows will first ask, then install, then ask if the user wants to reboot.

Under SP2, Automatic Update is enabled by default, and it finally has a mode that will reboot a user's system at a given time (say, 3AM). This will make it more likely that Windows users will keep their systems up-to-date, and cause more problems if your code isn't compatible with the latest changes.

Device-Driver Updates. Microsoft is also evangelizing hardware developers to register their newest drivers with Windows Update more quickly and more often, which may mean that your customers will update to the latest WHQL (Windows Hardware Quality Lab) approved drivers more often. If you rely on compatibilities with particular driver releases, especially with display drivers, you may run into trouble here.

Online Crash Analysis. When XP crashes, it asks if you'd like to send information on the crashed application to Microsoft. This data is used to improve driver quality, and you can get access to it for feedback on your own applications. While this isn't a new feature in SP2, now's a good time to integrate these analysis tracebacks into your own QA program, if you haven't already, before the 64-bit revolution hits.

Internet Connection Firewall becomes Windows Firewall. Windows XP SP1 introduced the Internet Connection Firewall, a software-based firewall which filtered traffic against some over-the-wire exploits, but was disabled by default. Under SP2, this becomes Windows Firewall, which is enabled by default. Online gaming publishers should test for compatibility with SP2, and study the Microsoft documentation on how to enable the needed port(s).

On to 64-Bit Windows

While the advent of x86-64 systems is going to affect 32-bit Windows XP, they'll be even more pronounced on Windows XP 64-bit Edition (XP64), when it ships sometime this summer or so. If you haven’t tested for compatibility with your existing titles, start now.

16-bit code is dead. The WinHEC presentations contained a big surprise: Microsoft Windows XP 64-bit Edition won't run 16-bit software. Moreover, it's not at all clear that Longhorn will do it either. In a 64-bit business strategy session, Brian Marr baldly stated 16-bit Windows support is going away, while Samer Arafeh, Software Design Engineer for the Windows Base OS Kernel stated "No 16-bit support on 64-bit Windows. Win32 APIs for 16-bit support are stubbed out to fail." Impact: If you still use 16-bit Windows calls, take them out now. This will have the biggest affect on installers--the last bastion of 16-bit code. Update to a true 32-bit installer, which will probably be far more reliable anyway.

In practice, 16-bit applications will continue to run in emulation, (e.g., Virtual PC), but this looks like a huge sea change for die-hard users of old games. Not answered at WinHEC: whether Longhorn will directly run 16-bit applications, or only do so in emulation.

32-bit code Just Works. In case AMD's advertising campaign somehow skipped your town, 32-bit applications run just fine on their 64-bit processors. That's the whole point of x86-64: Your existing applications will run just fine, and new 64-bit ones work better. Your existing titles may also get a 5-10% performance boost by running on XP64, because all the operating system calls will use 64-bit operations. You can test this today on the XP64 beta.

OpenGL is now a grudgingly accepted citizen. Since late in the NT 4.0 era, Microsoft hasn't wanted to support OpenGL, preferring to evangelize DirectX as the One True Way to display 3D. Between the graphics chip vendors doing their own OpenGL drivers, and the big CAD companies continuing to require it, Microsoft has put OpenGL support back into the operating system, so you can rely on it being there.

When to start writing for 64-bit code? There are a few game developers who have already begun porting their hot games to 64-bit Windows, specifically for AMD-based systems. Those titles will be touted even more by AMD when XP64 ships, as examples of the superiority of the AMD 64-bit architecture. They may even be coded to only run on AMD's CPUs, and not Intel's, at least for the initial release.

Everyone not in that NDA-protected camp will have to make their own decision. Fortunately, your existing 32-bit titles will run just fine on x86-64; the main questions are how quickly you'll need that extra boost in performance, and how much development time it will cost.

Compiler support. Microsoft Visual Studio is rumbling, slowly, toward x86-64 compatibility but probably won't support dual-platform development before early 2005. Much of Visual Studio is being designed toward .NET compatibility, which might not impress game developers much. The entire Common Language Runtime (CLR) interpreted-code approach is a polar opposite to most companies' fast-as-I-can approach. It's very unclear what "XNA", Microsoft's new game-development architecture, means, since it wasn't mentioned even once at WinHEC (I checked).

Obviously, Intel's compilers aren't going to support the Athlon64 or Opteron in 64-bit modes. Other compilers will, though the question is "when".

Installers. Microsoft's own installer, and those from the big-name third-party developers, have been architected to do conditional installation of either 32-bit or 64-bit code for several years, courtesy the Itanium, and appear ready for x86-64 as well. If you create separate 32- and 64-bit code images, the correct one will be installed. However, you'll probably want to check for the correct processor at runtime anyway, to avoid support calls from anyone moving their installed images between machines.

Middleware/tool support. Talk to your major tool, middleware, and game-engine partners--ask them if they've done any testing for x86-64 and XP64 compatibility. Do they have plans for supporting x86-64 as a separate platform? If so, when?

Your own development system workflow. If you already develop for more than one platform, you probably have a code-management/source code management tool that's savvy about multiple targets. Consider spending a little time now to add an x86-64 instance to your development system, even if you're not going to use it for several years.

The risk of un-hipness. Game developers, then, can take advantage of x86-64 well before Longhorn ships. You may be at risk for being thought old-fashioned if you don't release 64-bit versions of your hot titles along with everyone else--and that could be bad for your bottom line. The phase-change is going to happen far more quickly in the minds of the leading-edge gamers than in the general public's. Consider, then, migrating in parts; if you can get a 5% frame rate increase by porting your T&L code to x86-64, release that as a patch.

Longhorn and 64-bit Code

Longhorn will be of far greater impact for game developers than XP64--but not for quite a while. Few of its key features, particularly the graphically related ones, are going to be back-ported to XP or XP64, so until Longhorn is the dominant operating systems, most games will continue to support Windows 2000 and XP. Longhorn's graphics changes, and how they'll affect game developers, are a subject for another article.

Resources

Microsoft Documentation on Windows XP Service Pack 2 (http://www.microsoft.com/technet/prodtechnol/winxppro/maintain/winxpsp2.mspx)
A Technical Preview Program, release notes, and other background information on SP2.

Related PowerPoint presentations from WinHEC 2004:

64-Bit Business Strategy
(http://download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/SW04001_WINHEC2004.ppt)
Microsoft's 64-bit Windows client and server technology roadmap and strategy.

Porting the ATI Radeon's Direct3D Driver to AMD Athlon64 (http://download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/DW04056_WINHEC2004.ppt)
ATI's experiences in porting the ATI Radeon driver for Microsoft Direct3D to the AMD Athlon64 platform.

WoW64 Internal Architecture
(http://download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/DW04040_WINHEC2004.ppt)
Architecture of the Win32 emulation layer on 64-bit Windows, WoW64 process address space layout, registry redirection and reflection, file system redirection, 32-bit I/O on 64-bit Windows, and debugging.

Windows XP SP2 Configuration and Preinstall Considerations (http://download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/TW04094_WINHEC2004.ppt)
Implementation details for preinstallation and end-user installation experiences for Windows XP SP2.

Windows XP SP2 New Hardware Support (http://download.microsoft.com/download/1/8/f/18f8cee2-0b64-41f2-893d-a6f2295b40c8/TW04093_WINHEC2004.ppt)
New hardware support in Windows XP SP2, including Bluetooth wireless technologies and NX capabilities in newer CPUs.

WinHEC Vendor-Sponsored Papers

These Word documents come from Platinum/Co-Sponsor AMD:

Configuring Microsoft Visual Studio Projects to Support the AMD64 Architecture (http://download.microsoft.com/download/5/b/5/5b5bec17-ea71-4653-9539-204a672f11cf/VSProj_AMD64.doc)

Porting and Optimizing Applications on 64-bit Windows for AMD64 Architecture (http://download.microsoft.com/download/5/b/5/5b5bec17-ea71-4653-9539-204a672f11cf/AMD64_PortApp.doc)

Porting and Optimizing Multimedia Codecs for AMD64 Architecture on Windows (http://download.microsoft.com/download/5/b/5/5b5bec17-ea71-4653-9539-204a672f11cf/MMCodec_amd64.doc)

______________________________________________________