Zeigen

Continuing on the 1% theme from yesterday, the worst kind of bugs to deal with are critical bugs that occur both intermittently and rarely.

On the one hand, a bug that only happens 1% of the time may be okay. If you can easily recover from it, it may not be worth prioritizing. Imagine a web form, for example, where you hit Submit, and about 1% of the time, you get an error message that says, “Oops, something went wrong, please try again.” If you don’t lose any data and can simply hit the button again and it works, digging into the cause of the problem may take days. If you’re short on time, that kind of bug is often deferred.

But if it’s something more serious, such as a bug involving data loss, and it only happens 1% of the time, but isn’t reproducible (that is, if the same set of steps don’t always cause the bug to occur), then you’re dealing with a disastrous situation. Often times your beta program has not uncovered the bug. A beta program involving even 500 users is expensive and difficult to coordinate properly. During the course of your beta, if you’re lucky, you may see the 1% bug 10 times and prioritize a fix. But if you’re unlucky, you may only see it once or not at all. If it only occurs once (and the beta tester actually reports it), your beta coordination team has to be very sharp to notice the issue and flag it appropriately. Most of the time it will be overlooked, and even if it isn’t, it’s quite possible for a developer to defer the bug if he or she can’t reproduce the problem.

If you’re a mechanic working on a car, you’d much rather deal with a repair where there’s smoke coming out of the engine, because there you can see what’s wrong and know where the issue is. You’d rather not have to troubleshoot a problem where the driver says that sometimes the car stalls when shifting between first and second but only if it’s cold and only on Tuesdays. (That’s the kind of problem that is best suited for Car Talk.) The same applies to computer bugs.

Every program of any complexity that we use has bugs. I think users who aren’t involved with developing software often wonder why bugs are not all found and removed before a product is released. But the more complexity there is, the more variables, and consequently the harder it is for a developer to even experience the bug prior to release in the first place. (And developers spent enough time fixing the bugs they know about, they’re never going to fix the ones they don’t know about.)

One percent bugs are the most insidious, because they’re the easiest to miss or defer pre-release, yet once you roll out to the general public, it’ll quickly become apparent there’s an issue. Even if only 1% of users experience the problem, if you have a user base of a few million customers, then you’re talking about tens of thousands of support cases — enough to overwhelm the support center of even the largest of companies.

Good coding and unit testing practices can help, along with automated testing suites that hammer on the product at high rates of speed.

This entry was posted by Stephen on Wednesday, January 14th, 2009 at 8:12 pm and is filed under business. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.