In his 1984 book, Normal Accidents: Living With High Risk-Technologies, Yale professor Charles Perrow defined a normal accident as an unexpected yet inevitable failure that occurs when a given system becomes overly complex. More complexity leads to individual, seemingly unconnected parts of a system, interacting with each other in ways that no one could have foreseen. Perrow argues that even the complexity added by a system’s safety features — backup generators, alarms, warning lights, pressure valves, etc. — end up making normal accidents more probable because each of those safeties represents some extraneous moving piece that can itself break down.
One of the examples Perrow uses to illustrate the idea of a normal accident is the 1979 partial meltdown at the Three Mile Island nuclear power plant. Some of the factors were expected (leaky seals in the cooling system), others were mysterious (two valves in an emergency cooling system were mistakenly closed after a maintenance check), and yet others were seemingly random (an important warning light located on the plant’s vast control panel was obscured by a repair tag hanging from a switch above it).
A video game is, of course, a much less important system than a nuclear power plant, but the concept of complexity-driven bugs leading to broader failures is just as applicable. Just in the last two months, a number of big-name titles have been released with myriad hilarious or game-breaking high-profile bugs. Assassin’s Creed Unity had, among other problems, the strange habit of rendering some characters’ faces as a free-floating pair of eyeballs above a set of teeth and gums orbiting within a nest of unconnected hair. Call of Duty: Advanced Warfare wasn’t playable on PCs until about a week after its release. The racing game Driveclub went for weeks with its online functionality, well, not functioning.
Then there’s 343 Studios’s much-anticipated and technically ambitious Halo: The Master Chief Collection, which was released November 11.
Halo: MCC combines the full single-player campaigns and multiplayer modes of four previous Halo games in one overarching package, with graphical and audio upgrades meant to bring the games into the next generation. Included in the Halo: MCC are:
Halo: Combat Evolved Anniversary. This is the graphically enhanced version of the original 2001 Halo game, which introduced players to the laconic protagonist Master Chief and his AI companion Cortana, and showed that the console first-person shooter, then a niche genre, contained unexpected reservoirs of depth and creativity.
Halo 2 Anniversary. Where the first Halo revolutionized the mechanics of the console first-person shooter, Halo 2 shaped the way console games were played online. The innovative lobby structure allowed friends to stay together across matches.
Halo 3. The first Halo game on the Xbox 360, Halo 3 allowed players to design and share their own multiplayer maps. It also gave players powerful replay tools for filming and editing game footage, complete with reverse-motion, slow-motion, and free-flying camera angles, that could be used to make videos like this and this.
Halo 4. Feeling constrained by the demands of cranking out Halo game after Halo game, series creator Bungie split from Microsoft in 2007. In 2009, Microsoft created 343 Industries — named after Halo character 343 Guilty Spark — to take over the development of the series, then valued at about $2 billion. Halo 4, released in 2012, was the first Halo game developed by 343 Industries. Though Halo 4 sold well, 343’s attempt to co-opt Call of Duty–style game mechanics led to a muddled experience, and interest in the game dropped off precipitously in the months after its release.
Halo 5: Guardians (multiplayer beta). Halo 5 is slated for release in late 2015, and the beta test for the game’s multiplayer will go live as a download to Halo: MCC at the end of this month.
To recap: That’s four previously released games (plus a beta of an upcoming game) that were developed by two different game design companies to run on three separate video game consoles, all in one package.
The game code for just the first Halo game is some 14 years old, and encompasses a reported 1.6 million lines. Add to that the however many millions of lines of code used to build Halo 2, 3, and 4, the individual graphics engines that run each of those games, plus the remastered next-gen graphics and the unified navigation menu that sits on top of that legacy software. “We’re basically running two [graphics] engines at the same time,” Dan Ayoub, 343 Industries’s studio head for Halo external development told me over the phone ahead of the collection’s release. “We’re running the original Halo code, as well as our new graphics and sound engines simultaneously.” Then there’s each game’s multiplayer system, originally written to interact with the Xbox Live service as it existed on the date of the respective game’s release. And if that isn’t enough, the hardware itself, the next-gen Xbox One, is still in the first stages of its lifespan.
It is unsurprising, then, that the Master Chief Collection launch has been beset with numerous game-breaking problems, many of which seem to have taken 343 Industries completely by surprise. The core problem — but by no means the only problem — has to do with the game’s multiplayer functionality, a devastating failure for a franchise that owes much of its historical success to seamless, innovative multiplayer design. “We have to get the multiplayer right,” Ayoub said. “It’s just such an iconic experience, so memorable for so many people.”
Except no one could get into a multiplayer game. In the weeks after Halo: MCC’s release, players (myself included) would enter limbo-like multiplayer lobbies that could take hours — literal hours — to link with other players, at which point the game would invariably produce matches composed of uneven teams, with six players versus four or four players versus one. Halo’s numerous fans, many of whom took off work on release day in order to maximize their game time, were quite naturally displeased; a Reddit thread devoted to the game’s numerous bugs has more than 700 individual replies.
343 Studios head Bonnie Ross issued an apology on November 24 that read in part:
On the matchmaking front, we have encountered unexpected issues that were not apparent in our internal test environment and that have resulted in a frustrating experience, including long matchmaking times and low session success rates. Within 343 Industries and Xbox, I can assure you that resolving these issues is our #1 priority. We continue to partner with the Xbox platform team to analyze all data to make ongoing server-side adjustments to continually improve the matchmaking experience. We are also preparing additional content updates that will address existing campaign, UI, and other issues to improve the overall experience. With each update we will carefully analyze data to confirm that the improvements we’re seeing internally are also happening with fans at home.
It’s easy to blame 343 and the game designers for the game’s flaws, and certainly there’s no excuse for releasing a product that doesn’t work. But I wonder: Was the ambition and vision of the Master Chief Collection ever realistically attainable, considering the complexity of the project, without massively increasing the development and testing timeline? And was delaying the game ever a viable lever that 343 Industries could pull?
The pressure to release Microsoft’s flagship game IP on time for the holidays must have been enormous, especially considering Sony’s PlayStation 4 hit the 10 million units sold mark about three months before the Xbox One did. Even with a delayed launch, would it be possible to identify multiplayer problems without the unpredictable stresses of millions of players from around the world trying to connect all at once? You can try to simulate the height from which you can drop an egg without it cracking, but until you drop it, you just don’t know.
Certainly, if you listened to some of the things the developers were saying before launch, those concerns that the game might not be ready — and that they might not even be capable of knowing if it’s ready — were there. Max Hoberman, the original design lead for Halo 2 multiplayer under Bungie and now with Certain Affinity, said during July’s RTX gaming convention, “When they told me what they were planning when they first started this project, I told them they were crazy.”
The game works now. Kinda. “Now” being nearly a month since the game’s release date and after, by my count, three software updates, including a patch that had to be downloaded on Day 1. The single-player campaigns are all playable, though some bugs involving corrupted game saves have been reported. They are as fun to play as I remember them, and the two graphics engine systems that Ayoub talked about allow you to toggle the updated visuals off and on with the touch of a button. It’s a trip to see just how much of the way you experience a game takes place in your head, and turning off the shiny new graphics is a great way to take the measure of my younger self’s imagination.
The multiplayer — again, when it works, which it is now doing with more frequency — is a blast. You can select the Team Slayer feature, and the game usually rotates you through a random string of maps selected from any of the four original Halo games included in the package. The gameplay feels just the way I left it 10 years ago, but with added graphical shine and pop.
In our conversation ahead of Master Chief Collection’s release, I asked Ayoub to describe, from his perspective, the difficulties inherent in trying to build these incredibly complex things called video games. After all, even bad video games (and there are many) come from the labors of teams of impossibly smart, dedicated people just trying to do a good job.
“You know, I often feel amazed that these things get made at all,” Ayoub said. “It’s insanely hard. It’s this incredible genesis that needs to happen because you’re marrying such different disciplines together. You’re marrying these very hard technical problems and you’re bringing these together with artists who want to achieve something and designers who are trying to achieve something. And the really great games pop because all those things work in unison. But it is insanely hard to make it happen. If you look just at the Master Chief Collection, this was easily the toughest one that I’ve done in my career. The last few weeks, there were times I just felt like our heads were going to explode. There’s just so many moving pieces that you’re trying to grab and pin down, and you fix one thing that breaks three other things.”
A month after release, Halo: The Master Chief Collection is a less broken game than it was. And really fun. When it works.