In this article, I will discuss why so many new games are unfinished and buggy. A trend that leads to a tedious cycle of player frustration and downloading patches.
We’ve witnessed a string of recent game including Star Wars Jedi: Survivor, EVERSPACE 2, Redfall, Diablo IV, and Lords of the Fallen getting released in a broken state. So, I wanted to address an important topic for players, why we keep getting unfinished and buggy video games. This will tie in with my earlier article, The Top 10 Reasons Why Games Get Delayed.
Why are Patches Needed?
Patching video games started decades ago on the PC. It began by sending players updates on discs in the mail. Once the internet gave developers a portal to the players, more developers started offering optional patches to fix bugs that slipped through the cracks. Prior to the internet, if a game wasn’t compatible with your computer, you had to hope the store would take it back. The first patches only addressed critical bugs like the game being unplayable on hardware or crashes.
Let’s start off by saying that it is impossible to ship a video game that does not contain any bugs. I know that is not what players want to hear, but it is the truth. There are so many systems and features in video games that it is impossible to reproduce and fix every problem.
Patching on PC
PC games have a significantly more difficult time working on every combination of hardware that is in the wild. Game developers simply do not have the ability to create PCs that contain each hardware permutation available to find what works and what doesn’t. On most PC titles I worked on, we had one programmer who was dedicated to tracking down and fixing them. They usually had a large desk with multiple PCs containing a range of AMD/Intel CPUs and a drawer/cabinet full of video cards to swap in and out as needed. Unfortunately, there are far too many options on the market, so we tested hardware that was being used by the largest player base.
On top of that, we couldn’t predict what driver versions or other software was running on the player’s PC. We did the best we could given our limited time and resources. Patching was the solution to catching the edge cases and fixing problems. Sending the game to the graphics card manufacturers prior to release also gave them the chance to test and fix compatibility problems in the drivers before launch. A practice that most game developers do not follow.
Console Patching
Now let’s turn our focus to console patching. In late 2002, Microsoft released Xbox Live and Sony offered a network adapter for the PS2.
At the time, I was working on a sports game and we took full advantage of the service by using it to offer Xbox players updated rosters. These were small files that the game checked before trying to load the roster from disc. They were not changes to the game’s executable.
The next phase was to start updating online multiplayer games with Unreal Championship being the first to patch on Xbox. Online games were targeted first since the developers knew players were using an online connection. With the introduction of patching, game companies limited changes to updating the executable. These often included bugs that only showed up in multiplayer (desyncs), crashes, and code-related performance problems.
We entered our current environment once the hardware manufacturers gave developers the ability to fully patch games including the assets. Giving the developers this option allows games to be expanded after launch. In general, it is still discouraged to update large game assets that require massive downloads of 10GB+. But, this doesn’t stop developers from doing so.
Now that we understand the basic history of patching and why it can be necessary, let’s explore how this resulted in so many broken games being released.
Company Priorities
Ultimately, every company tries to release their products to make money, and game development is not cheap. Large, publicly traded companies need to keep the shareholders happy. Smaller development studios typically don’t have the cash saved up to continue making a game that takes longer than anticipated. In both situations, extended production cycles will lead to rushed launches.
In the early stages of game development, it is fairly predictable how long it will take to implement new features. Especially ones that the team has made for other games. Few teams take into account that an implemented feature is not done until it is fully tested. During this phase, the management team tries to predict when the game will come out.
Unfortunately, the process of getting a game ready to ship is not easy to predict. This results in teams committing to a launch date that they can’t make without sacrificing quality or being delayed. If the team is lucky enough to get the game delayed, it can only happen so many times before the company will tell the team they cannot move the date any further.
If the game is going to be released on physical media or have a marketing campaign tied to it, both require committed dates. Game disc manufacturers need to order hardware, prep for game boxes, and have space in their process to produce discs and ship them to stores on time. Creating and buying ad space also requires locked-in dates to succeed. Both of these factors often push a game out to the public that would be held back a bit longer for last-minute critical fixes.
Performance & Stability
The two most noticeable problems with recent games revolve around performance and stability. Performance is how well the game maintains a constant frame rate. If the frame rate throttles, it can be a jarring experience for players. Even more noticeable are stability bugs that lead to crashes or soft locks.
Frame Rate
Unless a game only targets high-end PCs, it is easier to get a predictable frame rate on console hardware than it is on PC. Given the countless ways we configure PC hardware, it is impossible to account for all these permutations when testing games. As a result, playing on a configuration that was not present on the test team may result in unwanted performance drops.
Having said that, there’s no good excuse for releasing a game on consoles that suffer from sustained low frame rates. The hardware for an Xbox Series X or a PS5 doesn’t change. So, the developers should verify that they are at or near their target frame rates for the majority of the time before shipping the game.
Lords of the Fallen was disappointing in this area. If you played the game for more than an hour, the frame rate degraded to the point where it was unplayable and needed to be restarted. The game never should have passed internal or 1st party certification.
Crashes
Crashes come in a few categories. Some crashes are 100% reproducible if you follow specific steps. Others may be caused by something that moves around like a memory leak and requires playing for a specific amount of time repeating a process in game. These can be a bit tougher to track down since you can’t simply follow a guaranteed set of steps. Then, there are the soft locks.
Soft locks occur when you notice music continuing to play while the game is unresponsive. This is often caused by an infinite loop in one of the main game threads which still allows the audio thread to update separately, but still requires a full restart.
All of these stability bugs are usually treated by game teams as top priority, critical bugs. The rare ones may only show up once or twice during testing (if at all) and likely would not get fixed. It’s not until you have 10k+ players that they rear their head and can finally be discovered and addressed.
Streaming Stalls
The final category of bugs I wanted to cover are related to streaming. Most games stream at least some of their content dynamically as you play. If implemented well, players should never notice it happening and question what they saw. When done poorly, we see low-resolution textures, objects or terrain popping in, and occasionally full frame stalls waiting for assets to populate.
Open Beta
You may be thinking to yourself, why don’t game developers just do open betas prior to release? Open betas are a two-edged sword. They can help game teams identify problems prior to launch at a cost.
Pros of Open Betas
Releasing a trimmed down version of the game to a smaller audience helps to identify some problems in the wild. Open betas are helpful with multiplayer games and PC games.
Multiplayer games benefit from getting a larger pool of players hitting the multiplayer services from around the world. Many networking problems can be simulated during development, but real world data is more valuable in solidifying the multiplayer experience.
On the PC side, getting a representative chunk of the game into more players’ hands early helps expose compatibility bugs. Giving players the option to upload their machine specs (dxdiags) can significantly speed up addressing these problems. Or, it can highlight when a player is running on outdated drivers.
Cons of Open Betas
Game teams don’t simply release an open beta to the public without paying a price. It takes time and effort to trim down the game to something stand alone that makes sense and does not contain files and assets that give away more of the game. Additionally, well-handled beta programs usually have a few dedicated staff members who maintain the beta build. These individuals help to track reported problems, and make updates to the beta to fix problems when possible. As mentioned earlier in this article, the end of a game’s development cycle is not predictable. So, pulling enough people off the main game in order to properly support a beta will result in some additional sacrifices of time (delays) or quality from the final release.
Fixing Bugs After Launch
If you’re like me and run into bugs while playing a game, you may check message boards and Discord to see if they have been reported or if there is a work around. Developers often check these discussions to see what they missed. If you want to ensure the developers address real problems, please try to post bugs that actually block the ability to finish the game. Discussions online tend to get bogged down with superficial complaints about how something looks or plays which is not likely to ever reach the bug database.
Critical Bugs
As mentioned earlier, critical fixes take top priority over all other bugs. These typically include reproducible crashes, progression blockers, holes in collision meshes, or licensing/legal content. A common legal mistake is using a red plus symbol for health. The symbol is actually a trademark for the Red Cross organization and cannot be used in games without permission. Critical fixes get prioritized over all other asks from the players after a game launches.
After a game goes live, it is much more expensive to fix bugs in the environment. Since the cost is high, most game teams will limit these changes to fixing problems with collision like holes where you can move outside the environment. Patching the world data will result in larger downloads for all players. If changes are going to be made, game teams try to do them all in one patch and lock down environmental changes after that.
Multiplayer
Bugs that only show up in multiplayer are usually the easiest to address. The majority of these issues will either be in the game executable or on the server. Server changes can be rolled out as soon as they are verified and a deployment plan is ready to execute. Game executable changes may need to wait. If the game is on multiple platforms and contains cross-platform play, all platforms need to pass certification before deploying the patch. If this isn’t done, you end up with cross-platform play breaking every time the developers touch the PC version before waiting on the consoles.
Alternatives to Patching
Patching the game itself isn’t the only way to fix problems once a game goes live. We already mentioned updating servers for multiplayer games. There is another option.
If designed early in development, game teams can setup tunable data to be verified online when the game launches. Doing so lets game designers make minor tweaks to balance data post-launch without needing to take the game through a full certification process. This can help fix noticeable problems with the power/cost of weapons/abilities either being too strong or too weak. As long as the change is a simple number adjustment in a data file, this can speed up the turnaround needed to keep players satisfied. Unfortunately, it is not a common practice and most balance changes end up requiring the game to pass through cert again.
In Closing
That’s it for the reasons why game developers continue shipping unfinished and buggy video games.
Thanks, everyone for taking the time to learn more about game development. If you have any questions or recommended topics for future posts or videos, feel free to post them in the comments below.