How to write good bug reports for game developers

Every game in development has bugs! And when discovered, someone has to describe the issue in the hope that it will get implemented or addressed, and the quality of that description makes a huge impact on whether this actually happens, and when it does, how effectively.

So what can you do to ensure that your bug reports are as effective as possible? I'll give you my personal perspective on what I want to see in a report when I'm the programmer asked to take a look at it. You won't always be able to do all of this, and that's okay. But the more you do, the better the report will be.

And the subtext in a lot of this is that good bug reporting is not just about what bug reporters need to do - it's what programmers need to provide to the team in order to make these reports effective. Read on to learn more!

Overview

Ultimately any good bug report covers 3 key areas - and in fact this is a good template for almost any request for technical help:

1. what you did
2. what you saw
3. what you had expected or wanted to happen

These aren't necessarily three separate sections of the report, and trying to write the report in this strict order is often too awkward. Instead, think of them more as three questions you need to have answered, or three areas you need to have covered.

Let's go through each one.

What You Did

Configuration

By this, we mean 'what setup or configuration did you have when you saw this'. What is relevant will depend on your project's situation but it could include the following:

• platform - is this on Windows, Mac, XBox, Android, iOS? Which version?
• hardware - what graphics card are you using? How much RAM do you have? What's your screen resolution - and how many monitors? Were you using mouse and keyboard or a gamepad? If it's a mobile device, which model? And which screen orientation?
• version - which version of the game were you running? Is there a build number, or date? Did it come from a specific branch in source control?

Ideally you don't want to be manually typing this out every time. It may be possible to use a 3rd party tool like dxdiag to gather most of the information, and then to copy and paste that in to each report as necessary. If you're using a bug database like JIRA, try and configure it to use sensible defaults and copy+paste in the rest.

State

What state was the game in when this happened? If possible, you want to share enough of the state so that someone else can try and reproduce the problem from the same starting point as you. This could be a save game file which someone can load up and experience the bug for themselves, or an exported character, or some other aspect of exported game state that can be loaded in later.

If you don't have any functionality like this, ask your engineers to provide this - it will help everyone!

Then, describe the in-game situation where the problem occurs, which might include:

• application state (main menu, connecting to servers, loading, play, pause menu, etc)
• 'macro' gameplay state (which level or map, in or out of combat, whether a special mode such as a cutscene or a conversation is active, which windows are open)
• 'micro' game state (character world location, character level and stats, currently equipped items, objects selected, buildings constructed)

What is relevant varies from game to game but the more information you can provide, the better. Programmers don't want to have to keep coming back and asking for more details because it holds things up.

Many games output some sort of log file that captures much of the above in text form. This can be attached to the bug report to answer a lot of these questions automatically. It's also good to get into the habit of checking the end of the log file before you attach it to see if there is any obvious error message in there which could be usefully pasted into the issue description. Often the cause of the bug is right there in the log files, and with some game engines it'll even highlight the exact line of code or the object that has the problem.

If you don't have your game generating log files, ask your programmers to set them up. It's likely that they already have some sort of internal logging system for their own use, in which case they just need to extend it to produce a file which you can send with bug reports.

Steps to reproduce

The final part of what you did is more obvious - this is where you provide the full instructions to reproduce the problem - "repro steps" for short.

Ideally these steps are comprehensive enough so that someone with only minimal familiarity with the codebase, such as a junior programmer or an external contractor, can still understand what to do. Bugs often get assigned to new starters as an onboarding exercise or outsourced to 3rd party studios, so a good bug report takes this into account.

Another ideal to aim for is a minimal test case - how can the problem be reproduced in as few steps as possible? Every step that can be removed will not only speed up the debugging work, but will also make it clearer which aspects are irrelevant to the problem.

For example, if you played through 3 rounds of a tournament and the game crashed at the start of round 4, is it possible that the same round would crash if you went straight into that one from the start? If you can try that round as the first one and it crashes, then the problem is likely to be specific to that round - but if not, the problem could be related to going through multiple rounds.

This is another area where programmers will usually have to provide tools such as debug commands or cheat codes so that specific parts of the game can be tested in isolation. These not only save time for bug reporters but for anyone developing or testing the game.

Finally, you want the scope of the test case to be appropriate. Often a hasty bug report implicates a very specific situation or object when in fact the problem turns out to be much more general.

Some examples:

• If the long sword is broken, try the short sword - is that broken too? If not, you have a long sword bug, If so, maybe it's all swords, or all weapons.
• If there's a visual glitch on your NVIDIA machine, does it occur on AMD machines too? How about with integrated graphics? You might need to rope in a colleague to help you check this sort of situation.
• If something is broken on the Dec 12th build, did the same thing work in the Dec 11th build? Can you find when exactly it broke? Or any build where it works? (Again, if you don't have access to old builds, ask for a process to be set up where you do have this.)

By finding the most closely-fitting scope for the bug report, you potentially save a lot of time when it comes to digging through the code and data to locate the exact problem. So, try some similar-but-different cases and adjust the report to include your findings.

What You Saw

Description

First, describe the situation in plain text. Use project-specific terms but avoid jargon or trendy terms gamers may know but game developers may not! Sometimes, even designers and programmers on the same team have a different name for the same concept or feature - a project glossary can help with this, so consider setting one up.

The aim is that anyone on the team should be able to read this description and understand the problem.

Visuals

Next, if you can demonstrate the issue with screenshots and attach them to the ticket, great. The Windows Snipping tool lets you quickly mark up a screenshot with arrows and circles to highlight the areas of interest. And if you can record and upload a video of the problem, even better. OBS will let you capture and export videos but the Windows Game Bar will do, or the AMD or NVIDIA streaming and capture software.

If you're wondering why the plain text description is still important when the problem is entirely evident from the videos or screenshots, there are a few reasons:

• Not everyone who looks at the bug report will have time to watch the videos or study the screen shots. A producer or team lead might want to just read the description to quickly understand who to assign the bug to.
• Screenshots and videos are not easily searchable. It's common to search through bug databases to see which issues are related or even potential duplicates of each other, and visual media doesn't help with that.
• Junior team members or people new to a particular part of the game may not fully understand what they are seeing in the video, or may not know why what they're seeing is considered a bug. The text can make this explicit for them.

Supporting Evidence

Often there will be some other documentation or evidence that you can provide which will potentially be a big help in diagnosing or fixing the problem. These include:

• pop-up message boxes with error messages
• console logs
• application log files
• error messages appearing elsewhere (e.g. debug text on-screen)
• crash dump files
• call stacks / backtraces

If your game uses an engine that allows you to run the game 'in' the engine, like Unity or Unreal do, it is a good idea to try and reproduce bugs in the editor, as this can often reveal extra diagnostic information that may be hidden in the final builds.

Metadata

Once you've detailed what you've seen, often there is something interesting to say about what you've seen too, drawing on your experience with the project.

A 'repro rate' is a useful thing to record - when you follow the steps to reproduce this, how often does it actually happen? Some bugs don't happen all the time, despite following the same repro steps. This is useful information to record, not just for the programmer who can know that it might be dependent on some other factors that are not yet understood, but also for a producer who can adjust the priority of bug fixes relative to how often they occur in practice. It's also a helpful guide to QA themselves - if a repro rate is low, perhaps there's a way to dig deeper and narrow down the cause so that a 100% rate can be achieved?

Another useful consideration is whether this bug is like a previous bug. You can search the bug database for keywords to find potential candidates, and if found, you can link the issues together so that the knowledge is shared, potentially making it easier to find the root cause. You don't have to necessarily work out whether two similar bugs are strictly duplicates of each other - just add the link in the database and let the coders figure it out.

What You Had Actually Expected

This is the part that confuses some people and which seems like a weird thing to ask for, but which is very important on larger and more complex projects. Here, you need to make clear how your expectations are different from what you actually saw.

Reference the specification

The main reason for this is that while most bugs are clearly a case of "this is doing the Wrong Thing", sometimes what you might think of as a bug is not strictly a bug, and is actually one of these:

• the intended behaviour, where a designer wanted it that way, for some reason that is not apparent
• behaviour the design team don't care about, and isn't necessarily wrong or right
• a situation that was never anticipated or specified at all, and needs to be addressed with a feature request rather than a bug report

So this is your chance to use both your expertise of games and your knowledge of the project to explain the situation. Why did you expect this behaviour? Is there a specification or design doc that shows unambiguously that this is wrong? If so, quote it and link it in the bug report.

This is a key reason why developers should not skimp on design documentation. Not only does a feature specification tell the implementor how to make the feature, but it provides a baseline that QA can test against when seeing if the feature is implemented correctly. And, for our purposes here, it provides a justification for logging a bug plus also a baseline to measure whether the resulting bugfix has correctly resolved a problem or not.

Reference external or implicit standards

Sometimes there are also external benchmarks or standards you will need to adhere to, for example age ratings (e.g. PEGI), technical platform requirements (especially on consoles), accessibility laws, etc. These can also be linked to when filing bugs, to make it clear what a fix would need to take into account.

But sometimes, you don't have a specification, and you can't get a ruling from a designer or producer - but you still think it's worth addressing. What then?

If it's a personal feeling that "this shouldn't be like this", you can try and justify it by de facto standards, such as comparison to other parts of the game ("this doesn't happen in the similar situation on level 3, so it shouldn't in level 4") or at least compared to standards in other games ("in similar games in our genre, double-clicking this icon normally restores the counter to zero").

A final justification - and some might say the most important one - is the impact on the player, since a player expects to not be frustrated by a game. What will a typical player feel if they hit this issue? Given that bugs can run the gamut from 'minor visual glitch, unlikely to be noticed by many', all the way through to 'accidentally overcharges the user $1000 when buying an in-app purchase', the level of impact can be the most decisive factor in whether this issue needs to be addressed immediately or can be kicked into the long grass to be fixed later - or never. If the game is already live you can sometimes include quotes from players or links to comments they've made, and this can help the team to assess the priority of the bug and sometimes even to gather more test data.

Make expectations clear

One last important thing to remember is that, from the information above, it needs to be clear what the result of a fix for the issue would look like. If there's agreement that there is a problem and work needs to be done to resolve it, the developer needs to know what the 'definition of done' for that work is, and QA need that information to be able to assess the fix.

Coders are often a surly bunch and having their well-intentioned bug fix re-opened because the initial information was lacking only makes them surlier! So make sure that they're fixing the right thing in the first place. This is another reason why written design specifications are important and why the team needs to be cognizant of the difference between a missing feature and a bug. Being asked to repeatedly fix something which was never technically broken but which was never adequately specified in the first place is a real morale-killer for programmers.

What Not To Do

Generally, more information is always better! But unless you've been asked for the following, don't provide them:

• Estimated fix duration or difficulty - unless you will be doing the work, don't feel tempted to estimate the size of the task. It will be inaccurate and could give a false impression to producers regarding how much time is needed to clear bugs.
• Who you think should fix it - often it's possible to guess who the best person for the job is, but assigning it to them is best done by whoever schedules tasks. It's fine to mention topic experts in the text so the information is there when it's assigned, of course.

Finally, don't re-open a past bug unless you're 100% sure it is exactly the same problem and caused by the same circumstances. There is a tendency for bugs with broad descriptions (e.g. "Game crashes to desktop", "Equipping item doesn't work") to get repeatedly re-opened as there can be several different problems or situations that can cause similar outcomes. But a re-opened bug implies that a previous fix has stopped working, which may not be the case if the problem has a new cause. Again, it is bad for developer morale to keep having your bug re-opened when your fix was actually fine, and it can also slow down diagnosis of the real problem because attention is diverted towards the previous fix rather than the new circumstances.

Instead, create a new bug, and link it to the previous one that you think might be related. This captures the relevant information in the system for the person allocating the tasks and the person performing the task, without prematurely implicating prior fixes or second-guessing the actual fix required.

In Closing

There was a lot covered above, and it might seem like following all the advice will make reporting a bug an onerous task, which in turn could lead to fewer reports and therefore a worse game. This is a legitimate concern as you want everyone to feel confident to log what they see and to be contributing to the quality of the game without it becoming a drain on their time.

So the important thing is to get into a position where you can do most or all of the above effectively and efficiently. Ideally this means having:

• Automated gathering of system specs, config files, build numbers, log files, etc
• Comprehensive debug tools and functionality that allow bugs to be reproduced quickly and easily
• Tools, scripts, or documentation on how to export a save game and load it on a different machine
• Access to good screen capture tools for stills and video, ideally with annotation capabilities
• Written specifications that describe how the game should act, look, and feel

With that in place, you should be well on your way to logging some effective bug reports that make a real difference to finishing the game on time and under budget!