Feature: 'Fast File Loading (Pt. 2)'

In the second of a two part technical Gamasutra feature, Pyro Studios (Commandos series) tech lead Jesus de Santos Garcia presents a guide for loading streaming data quickly off of media, such as a computer's hard drive. In the excerpt, de Santos Garcia describes the intent of the article, which covers techniques for organizing and loading data in a fast and efficient manner, as well as introduces an efficient processing technique called Load-In-Place: “In the previous article on Fast File Loading, I described techniques directly related with the hardware and the operating system to load files in the most efficient way. In this one, a technique for organizing the data to be loaded as fast as possible is described. Basically, we are going to load data without doing any parsing at all. Although the techniques described here are oriented towards real-time applications, any application could use them to accelerate its load times.” He later adds: “The basic philosophy under Load-In-Place is “preprocess as much as possible“: do not waste CPU ticks with operations that can be done offline, do not waste CPU ticks so that the the loading process can run in parallel without interfering with your main process. Imagine that you could load your data with a simple fread() from a file. Without doing any parsing at all. This is basically what we are going to try to do without modifying the structures we usually have. If we had, for example, a data file with a vector of floats, loading with this philosophy in mind would be as simple as memcpying the file to memory and pointing a float* to the beginning of the memory buffer. And that would be all. We would have our data ready for being used.” You can now read the full Gamasutra feature for de Santos Garcia's full description of a technique for loading data without parsing, including full Microsoft Visual Studio 8.0 project code (no registration required, please feel free to link to this column from external websites).