My parents say I was a noisy kid. It bugged them, often. However, now that I am an (much quieter) adult, they confided that they always were most-worried when I was quiet. Apparently, when I wasn’t making noise, it was because I was in the process of hatching an evil plot or something. I think this is funny because, when I’m programming, I’m quiet most of the time. Bwahaha!
Well, most of my programs don’t have evil outcomes. Probably the ones that were the most evil were only that way because of a serious bug or two. Yep, if you post your updates and they take down a web server that has been running nicely for a year or two, people aren’t going to have nice thoughts about you.
The thing that has kept this kind of mayhem to a minimum is: solid testing. A good test should be able to shake the big bugs out of a program (or an update). The key word here is good test. A weak, half-effort is less-likely to be effective, but it is still better than nothing. Maybe.
So, how much testing is needed for it to be a good test? Lets face it; no plan is ever perfect. For starters, it should seem pretty obvious that a plan is more perfect if it is more detailed and takes into account more factors and facets. Right? If you thought of everything, then you should be able to catch any/every bug.
So, can you think of everything? The reason I ask this is because, all of that detail also tends to make a plan seem more tedious and rigid. If one little thing changes, you may have to rewrite significant parts of the plan. That is time consuming, even risky to some degree. Also, if I make an extremely detailed plan and hand it to some testers, they are going to be a little miffed that they were no-longer given the opportunity to think for themselves, because the plan already did all of it for them.
The flip-side of this issue is “value”. Testing for EVERYTHING starts to seem pretty wasteful and even silly once you cross some lines. “Okay Dan, keep watching out of the window to see if it rains. The radar image might be flawed. We just can’t take that chance.”
I once had a discussion with my boss about this. She brought up a good point: “if you don’t make any mistakes, then you weren’t moving fast enough”. I had to think about that one for a moment. When I am rushed, I know I am more likely to make mistakes. The more rushed, and panicky, the greater the chance for mistakes. However, I never really thought about the balance between efficiency and mistakes.
Certainly, testing a program before you launch it is a great (probably even necessary) way to catch simple mistakes. The more time you spend testing an app, the more likely you are to catch any flaws. However, after a point, your return on investment starts to drop-off. In theory, you could spend a lifetime testing an app and still not catch all of the flaws in it. The key to getting good ROI on testing is to pick an amount of testing that is “just right”. To some degree, you won’t really know if you’ve found it until you cut back too much and a bug sneaks past you.
That can be a hard concept to swallow. Every programmer, in his heart, wants to make good software that runs well. Bug-free products are everyone’s goal. However, in this capitalist economy, there are plenty of examples of companies who waited too long to release their product and got passed-up by a competitor. On the opposite end of the spectrum, there have also been plenty of companies that delivered some really buggy crap and did major damage to their reputations. Fortunately, most of those companies were able to dig themselves out of their mess and repair their reputations.
Many people predicted that Microsoft’s buggy software would be their undoing. Microsoft certainly had to do a lot of hard work to repair their reputation, but their competitors (Novell, Apple, Sun, etc) were seemingly standing-still, making flawless software and getting smoked by Microsoft. So, when you think about it, the safer choice (from a business perspective) is to err on the side of being quick. You just need to be ready to adjust and fix any messes that you create.
So keep in mind, when things are quiet around your shop, too quiet, you might just be making too few mistakes. Push yourself out of your comfort zone and try spicing things up a little. Just don’t let it get out of hand.