Bug reports that fail

If you are a software developer, and you have ever worked with a new tester or a  “tester” (*ehem*) with no formal experience in software testing, then you know it is not fun at first.  If nobody teaches him/her how to do this job effectively, then it becomes your responsibility.  Oh sure, that person can find bugs alright.  However, communicating the bugs, to you, is a whole different story.

You can either train the tester about effective communication, up-front, or spend arbitrary amounts of time periodically, training the tester about the value of a [clear and useful] bug report vs. the time wasted while you try to solve his/her arcane riddles.

So, for your entertainment pleasure, I give you of my favorite examples of terrible bug reports.  These are taken from ACTUAL bug reports (changed slightly, to protect the identity of the projects/testers).  Just in case you are not clear about what is wrong, I also added a brief description for each.

  1. Program errors out – Problem: Vague, unrepeatable.  I could wander around the app for hours without finding the same thing that the tester found.  A description of what was being done to trigger the error, is necessary.
  2. The requirements were missed  – Problem: I know that I didn’t miss all of them, but there is no mention of which one(s) got missed.  In fact, there might not be any missed-feature after-all.  I have seen bug reports like this which were merely a matter of opinion or someone trying to recommend a new feature.
  3. Report alignment is off – Problem: Just because you think it is wrong, does not mean that I will know what “right” is supposed to look like.  It is better to be clear about what it should look like if it was right.
  4. I can’t log in – Problem: lack of permission is not really a bug.  Maybe you need to ask an admin for permissions. Otherwise, you need to describe the difference between your permissions and why you think they should be working differently.  Imagine the catastrophe that I could create if I “fixed” this bug, but it wasn’t really a programming flaw.  Your permissions would be “fixed” for you, but “broken” for everyone else. So, please reassure me that the program is not simply blocking you correctly (based on the permissions that were assigned to you).
  5. The internet is broken – Problem: Sorry, this is just not even possible.  Nobody can write a program to break the internet.  Maybe your connection is having a problem, or you cannot connect to one web site.
  6. My system crashes – Problem: “system” usually means your “operating system” but that is probably not what you mean.  You need to be clear about what system is crashing.  You also need to be more clear about what you mean by “crashing”.
  7. Colors are gaudy – Problem: there is no mention of criteria for gaudy or a explanation of how to resolve the problem.  This sounds like a simple difference of opinion.  If it is a real bug, I will need more proof, and corrective action(s).
  8. Profanity in the database  – (I actually fell for this one): I know that I DID NOT put any profanity in the DB.  However, I searched the database anyway.  I could not find any profanity. I finally asked what profanity to remove.  It was the Yiddish word for p*n*s.  I’m not Yiddish and neither were any of the people on my team.  So we had no idea what to look for. Problem: vague description made it unlikely that we could find/fix it.  As a result, we burned a lot of time looking for the problem.
  9. Program should not error-out – Problem: (it is mostly a repeat of #2 but with a strange twist) This was worded as if the program was violating a business rule.  If this was an actual business rule, it would be completely silly.  I just kept laughing about that wording: like, duh! “Oh golly, I nearly forgot that the program is NOT supposed to error out.  How did I miss that requirement?  Well, I better go fix all of the places where I implemented that business rule wrong.  Derr!”  If the tester could not produce a useful description (sometimes all you see is an error), then he should at least describe the list of steps to reproduce the problem.
  10. Query result is blank – Problem: no mention of query being used or the expected results.  The query had always produced results for the dev team. Let me check.  Yep, it still gives me results.  I could be equally obtuse and mark that one as fixed.  Instead, I have to ask you for these details.  It would be better if I didn’t have to ask.  I hope this pain doesn’t continue much longer.

Here are the most important rules of writing-up a bug report:

  1. Make sure you describe the steps to repeat the bug.  Anybody/everybody should be able to recreate the error from your description, and nothing else.  If I hand that bug report to my boss, he should be able to recreate it without asking anyone to clarify anything.  Otherwise, I might decide to be passive-aggressive and hand my boss your error report.  Then after watching him struggle, I will give you full-credit for it, of course.
  2. If there is any way that a developer could be un-clear on what “correct” should look like, then you need to spell it out.  If it is completely obvious and no reasonable person could be unclear, then do not be silly by describing something so obvious. For example (1): “Bug report: The box is red”. I would guess that it should be white (because white is not red), but if my guess is wrong, then this description has narrowed-down the possible correct colors by almost nothing.  There are a lot of colors that are not red, but which color is right?  Example (2): “Bug report: you spelled Wershington DC wrong”.  Golly, I should probably slap myself for that mistake.  When you say Wershington, you really mean Washington.  Unless you don’t, and then I will make an assumption.  It is better if I am not invited to make assumptions.

In the end, what we all want, is to get this stuff right.  So if you are a software tester, please make a point of being clear about what “right” is supposed to look like. You will add a lot of value to the team and we will all get there much sooner.

About Tim Golisch

I'm a geek. I do geeky things.
This entry was posted in IT Horror Stories, Lessons Learned, Testing. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s