Product SiteDocumentation Site

6.6. The Anatomy of a Good Bug Report

Take a closer look at the bugs in your project's bug tracker. In particular, spend some time looking through bugs in the new state.
It's certain that some of these bug reports are not be very useful.
For software to improve over time, good bug reports are absolutely essential. Users who file bugs, while almost always well-meaning, frequently don't know how to write good bug reports -- which is a problem, but also an opportunity for others to make positive contributions. Turning a bad bug report into a good bug report doesn't require special developer knowledge; it just requires patience, persistence, a solid understanding of how the software works, and an understanding of what good bug reports look like.
So what does a good bug report look like? Look at some of the characteristics of good bug reports.
Good bugs have descriptive summaries
Bad:		Audio player hangs
Good: 		Audio Player stutters when playing some kinds of audio files
Best:		Audio Player stutters when playing .mp3/.wav files > 35Mb, patch attached
Software developers, like most people, are busy and lazy in roughly equal parts. When a developer has decided to spend a day "fixing bugs," that developer likely starts by scanning the bug list. If she see a bug summary that says "your program is broken," it's unlikely to provoke a favorable response. Summaries matter. The goal is to pack as much information into a single line of text. The more information the developer sees in a bug summary, the more likely he is to dig into that bug.
Good bugs refer to the version of the software
Bad:		Zombie-buster version... two-something?
Good:		Zombie-buster-1.2.4
Best:		Zombie-buster-1.2.4 and confirmed in HEAD, svn checkout 1361
Most of the time, the bug tracker you use has a predefined field for version. That field is a good start, but sometimes the entries can be outdated, and even if there's an entry for HEAD (i.e. the very latest code in the SCM), it's still important to provide the most detailed version information that you can possibly provide. What you absolutely must NOT do: ignore this field. If you can't figure out what version of the software you're running, then you probably shouldn't even file a bug.
Good bugs provide relevant system information
Bad: 		I'm running Linux, if that matters.
Good:		I'm running Gnome 2.28 on Fedora 12.
Best:		Gnome 2.28, Fedora 12, nVidia Corporation G70 GeForce 7800 GTX (rev a1)
It's easy to provide way too much of this kind of information, but that's a forgivable sin: it's certainly better than providing none at all, which is too frequently the case. Over time, you develop a feel for which information to provide -- and if you become a trusted contributor to a project, developers feel more comfortable asking for precisely the information they need. Until developers say otherwise, it's usually best to err on the side of providing too much info. (Within reason. Dumping the entire contents of gigantic log files into a bug report is clearly bad form, but a relevant log file may be attached to a bug report.)
Good bugs only report a single issue
Bad:		When I load the latest Foomail client, it crashes.  Also, I'm getting screen lockups \
                in Barchat.  I think they're related.  
Good:		Bug 1: Foomail client crashes.  Bug 2: Lockups in Barchat.
Best:	    Bug 1: Foomail client crashes.  I think it may be related to lockups in Barchat \
                (and here's some evidence).  Bug 2: Lockups in Barchat.  I think it might be \
                related to Foomail client crashes (and here's some evidence).
It's tempting sometimes to see multiple issues, convince yourself that they are related, and then file a single bug on the whole thing.
Don't do that. You may very well be right that the issues are related -- but you may also be wrong, and in either case, two different issues deserve two different bug reports. In our example above, it may be that the issues with Foomail and Barchat are actually related to a problem with Bazlib, which is a shared component of Foomail and Barchat. Each bug must be fixed and tested in its own context, and that won't work if they're all stuffed in the same bug report.
Good bugs are followed by people who care about a solution
Bad:		What, you said file a bug, I filed a bug.  I don't care if you fix it.  How do I turn off these email notifications?
Good:		Hey, I checked the latest version like you asked, and it still isn't fixed.  Do you need more info?
Best:		I kept poking at this bug, and this log entry makes me think it's somewhere in libfoo.  I'll keep digging.
Never underestimate the power of attention. The more you care about a particular bug, the more likely it is that the developer will care about it. The more work you do to chase it down, the more obligated the developer feels to spend some time on it also, and the more appreciative he or she is. If you file a bug that no one cares enough to follow up on, not even you, then why should the developer care enough to fix it? Especially if there are other bugs that people do care about?
Good bugs are reproducible with the fewest possible steps
Bad:		start it and click around for a while and then it crashes
Good:		Start the application.  Click the mouse button and type on the keyboard.  Within \
                45 seconds or less, the application will always crash.
Best:		Start the application.  Click the mouse five times, and the application crashes with \
                the following error message in "/var/log/foo.err"...
Anybody can break software. People break software all the time. But it takes a bit of practice and skill to break things predictably, over and over -- and if there's no one exercising this skill on a project's behalf, it's much more difficult to improve that project over time. That's why software companies hire testers (although they almost never hire as many as they should.)
Reproducing bugs is one of the best ways to learn how software actually works. A tester who has explored a bug thoroughly, and who can confidently say "this bug only happens when these three conditions are met," is much more likely to be able to take the next step: namely, to dig in and find the code that actually creates those three conditions.
Sometimes it's not possible to reproduce a bug 100% of the time. Intermittent bugs are extremely frustrating for both testers and developers; they are frequently the result of complex software interactions, and reproducing the bug is actually the hardest part of fixing it. In such cases, testers should do their best to provide as much information as possible, and be ready to answer a developer's questions.

6.6.1. Exercise - Reproduce a Bug

Go through your project's bug tracker and find a bug that you think you might be able to reproduce -- and then try to reproduce it in the latest build. Take careful notes. Report your experiences as a comment to the bug. If you can reproduce the bug, great! Give as much information as you can. If you can't reproduce the bug, great! Give as much information as you can, and ask the original reporter if there are other steps you might be able to take to reproduce the bug.