How asset streaming is - and isn't - enabling new ideas for game design

It may seem like damning with faint praise to say that the best part of 007: First Light was its tutorial, but the training montage of IO Interactive's Bond simulator was arguably the standout sequence in a game that wasn't short of spectacular moments. The level's rapid-fire cuts between driving, shooting and sneaking within MI6's Maltese training facility managed to be instructive, thrilling and novel in the same moment, proving in five electric minutes that IO had the cinematic eye to handle 007.

But the sequence also shone a spotlight on one of recent history's more significant technologies: asset streaming. The last half decade has brought this method of data streaming to the fore, as the proliferation of solid state drives (SSDs) dramatically reduced loading times and facilitated rapid world and level changes. Alongside First Light's training montage, asset streaming has been held responsible for other snazzy sequences, particularly on PlayStation 5 where an SSD and "instant loading" were heavily touted features, such as the rapid portal transitions seen in Insomniac's Spider-Man 2 and Ratchet & Clank: Rift Apart.

Yet while asset streaming may be the technological poster child of this console generation, the technology - and data streaming in general - has long played a central role in how games are made. At the same time, there's more to these flashy set-pieces than fast loading times. Understanding why is important at a time when the relationship between creativity and technology is being scrutinised more intensely than ever.

But first, and at the risk of sounding like a primetime BBC documentary presenter, what is asset streaming? In layman's terms, it's where games load digital resources from your storage drive into memory as and when they are needed. "There's a game world, and there could be hundreds of different entities that take up the world, and those entities are assembled via different pieces of information," says Autumn Willow, senior game designer at Supermassive Games. "To render it in the software, the GPU needs to store in memory the addresses of all these different things … this where you get the idea of RAM. You can only remember so much, and modern game engines have ways to mitigate that, but at the end of the day, there is a hard limit."

"New technology, it's really only worth what it can do to enable creativity"

Asset streaming essentially ensures that your PC only needs to hold in memory the relevant data for that particular moment of whatever game you're playing. It's the reason open world games can present players with such vast landscapes without their computer melting. A game like Elder Scrolls V only needs to load a specific chunk of Skyrim at any given moment, rather than the entire world. 

But asset streaming is also an umbrella term for numerous technologies that have developed gradually through game development history, all of which revolve around shunting different types of data from storage into memory. One of the earliest examples relevant to games is texture streaming, which essentially does what asset streaming does, but for high-resolution textures rather than entire objects.

"Texture streaming is a technique that effectively increases the available console memory by not keeping textures (the color map, normal map, and specular map) in memory when they are not currently needed." says Robert Field, who was an engineering lead at Infinity Ward from 2003 to 2009. "As long as a lower-resolution version of a texture remains in memory as a backup, the worst-case scenario for an unexpected need for that texture is just a temporary blurry image." You've probably played a game where entering a new area presented you with soupy, low resolution textures, before the high resolution versions popped into place.

Field says that this makes it easier for game designers to manage assets within a game level. But it also has important knock-on effects for what those designers can do with a level. "It allowed designers and artists to improve the game detail and increase the size of continuous environments in the game," he says. "Unlike more linear environments, Call of Duty multiplayer maps had to handle the player being in different locations after respawning, though the game could preload textures for the respawn location."

Texture streaming can be traced as far back as 1996's Crash Bandicoot. But it, and data streaming more generally, became more prominent in the noughties as multiplatform releases became commonplace, and developers needed to find ways to bridge the gap between different consoles and PC. 

During his time at Infinity Ward, Field worked on and alongside various streaming solutions. On Call of Duty 2, Field built a fast level loading system called FastFile. "Before Call of Duty 2, the game used a PC file loading system which would have made a 30 second load time take minutes on console," he says. "[FastFile] worked by packaging a level as a single file to be loaded, such that the single file was loaded at the maximum bandwidth of the DVD or Blu-ray." 

On Modern Warfare 2, meanwhile, Field implemented full 3D texture streaming, which lowered the base memory requirement for a game level, thus giving designers more memory to work with. "It enabled Call of Duty: Modern Warfare 2 to have more detailed and larger environments compared to Call of Duty: Modern Warfare on both the 360 and the PS3," he says.

You've likely seen what happens when this work doesn't get finished. In a video apology recorded after Cyberpunk 2077 first launched, CD Projekt Red's then-CEO Marcin Iwinski blamed the "in-game streaming system" for its poor performance on the previous console generation, machines which did not have SSDs. Even on PC, if you saw characters T-posing during Cyberpunk 2077's own (non-interactive) montage, it was the fault of asset streaming that couldn't keep up with Night City's high resolution characters.

Today, most game developers, and the software they work with, take data streaming into consideration as a matter of course. At Supermassive, Willow works with Unreal Engine, which does much of the heavy lifting on the asset streaming front by default. "Unreal comes prepackaged with a whole bunch of things that make asset streaming kind of automatic," she says. "If it's on screen, it's going to try and load it into memory."

Better technology in and of itself does not offer solutions to problems

This doesn't mean all the hard work is done for developers, however. Supermassive, for example, makes highly cinematic games, which means the placement and detailing of objects is very important. Hence, there are times when Unreal's general solutions are insufficient and the designers need to intervene. "When we get specific frames or camera cuts or segments where we absolutely need something to be in memory to its highest capacity at a very specific point, that's when we get involved," Willow says. "You can do things like tell specific assets to be loaded. You can load them in the background, or you can force the game to stop and not progress until they're loaded, which is very useful." 

The introduction of SSDs has basically made this process significantly faster, meaning developers can make bigger, more detailed games with less segmentation than was possible a decade ago. "You get into these games now that are loading tons of assets, entire worlds, I guess the word would be asynchronously," Willow says. "While things are happening in the background, or just over a completely imperceptible amount of time, you can load in things that would have required a loading screen ten years ago." 

It's the latest in many steps that have led us from traversing linear corridors to exploring seamless open worlds. But, as mentioned, we've also seen more specific, creative applications of asset streaming in recent years. Insomniac had repeatedly showcased the power of the PS5's SSD, such as the Spider-Man 2 mission 'Make your own Choices', where Miles Morales chases Black Cat through a sequence of portals that transport them instantaneously to different parts of the globe. "I'm sure that Insomniac had the keys to the kingdom in terms of access to PS5 tech," Willow says. "They built that right into the game."

And then, of course, there's First Light's training montage. This is where I discovered that I began researching this article on what might have been a faulty premise, as Willow isn't convinced that this scene is necessarily dependent on asset streaming to function. "The island it's set on, it's all this sandstone and prefabricated spy assets, [and] any instances of it are just going to be always in memory because the whole island is made up of it," she says. "The agents involved, Monroe and Cressida, they're in every single scene, so they're always in memory—they've just got to be teleported around."

Instead, Willow suspects that this scene was made possible by the particular way that Hitman's Glacier Engine functions. "I believe Glacier can do a funny little trick where it can preload multiple viewports of the game. I think Hitman does this in a few places," she says. "If I was a betting woman, I would say what's happening in that montage is they're going into a scene, and then at some point during that scene, they're sort of shouting ahead to the next scene, going 'Hey, James Bond just finished the fighting, [so] I've ended the scene that's going to end this little scene-let—start loading the shooting range.' And then, when the camera cuts, it's already loaded."

Willow stresses that she hasn't worked with IO or Glacier, and so her viewpoint is only an educated guess. (We contacted IO Interactive for this article but they weren't able to participate.) But she also argues that whether the montage relies on asset streaming or not is a moot point. What really matters is that someone at IO had an idea for that montage and made it work: "Technology gets better and better," she says. "But at the end of the day, if you don't have people behind the game with a vision of what they want to do with that, then you won't see [many] changes." 

And like asset streaming itself, this idea is not new. Field points out that Modern Warfare 2's 3D texture streaming was built specifically facilitate the existing design, rather than the studio creating it and then figuring out what they could do with it: "The design for Call of Duty: Modern Warfare 2 was anticipated and desired beforehand to feature bigger and more detailed environments; however, the method of achieving this still needed to be worked out."

In short, better technology in and of itself does not offer solutions to problems. Indeed, it can actually create problems. Willow points out that, as data storage gets faster, developers "can have more detailed textures and larger static meshes." But developers still face the same hurdles in creating those textures and meshes, which become more intensive to make if you want to take advantage of the raised technological ceiling.  This is a major factor in why, despite hardware becoming faster and gamedev software more accessible, blockbuster games take longer to make now than they did ten years ago. "A fluid will fill any container it's put in," says Willow.

Willow believes this is likely to become more of a problem as generative AI takes root in the games industry, because AI seeks to remove problem solving from the creative pipeline entirely. "The technological movement at the moment with AI and DLSS 5 is to just get to that endpoint, to have the final product of creativity without doing all the work and investigating other technologies that facilitate that process," she says. "[But] the act of the craft itself informs you about different avenues you can take."

Creative offshoots like First Light's training montage, meanwhile, are exciting because they emerged from an interrogative relationship with technology. Someone at IO had that idea and grappled with the problem of how to realise it. "New technology, it's really only worth what it can do to enable creativity," Willow says. No matter how fast SSDs get, they'll only ever be as interesting as our imaginative capacity to use them.