-
Original address: Why bugs don't get fixed
Author: Alan Page, Director of Excellence in Test Engineering at Microsoft, co-author of the book How We Test Software at Microsoft (Chinese translated as "Microsoft's Software Testing Way").
Translation: Lu Yueli, Lu Mengyan, Wang Hong
Lately I've been encountering more and more people who are surprised that we would release a buggy product. What surprised me was that many of these people were software testers, and I thought they would already know about this. I suggest you read Eric Sink's earlier (but excellent) article first. Not sure how much more I can contribute to this topic, but I want to try.
Many bugs are not worth fixing. "Are you a tester?", you will scream at me, "Testers are the guardians of product quality." I can repeat it again (if necessary) many bugs are not worth fixing. "Let me tell you why. In most cases, fixing a bug requires changing the code. And changing the code requires an investment of resources (time) and introduces risk. That sucks, but it's true. Sometimes, if The risk and investment far outweigh the value of fixing the bugs, so we won't fix them.
Our decision about whether to fix a bug is not, and should not be, based on "feel". I like to use the concept of “user pain” to help me make decisions. I use three key factors to consider and identify "user pain":
1. Severity - What impact will this bug have - will it crash the entire program? Will it cause the user's information to be lost? Or is it not that serious? Is there an easier solution? Or is it just an irrelevant issue?
2. Frequency - Do users encounter this problem frequently? Is it part of the program's main workflow? Or is it hidden in a feature that is not commonly used? Small problems that exist in the most commonly used parts of the program will probably need to be fixed, while some big problems that exist in the less commonly used parts of the program may be put aside.
3. Impact on customers – If you’ve done your homework well, you should already know who your customers are and how many (or how many you hope to have) users there will be in each of your customer groups. Then you need to judge whether this problem will affect every user, or just some people. If you can track how customers use your product, you can get more accurate data.
The above three factors constitute a formula. Assign a range of values to each of the above factors and do some calculations - you can directly add, multiply or add weights based on your application and market factors. For example, we only need to perform addition and assign a numerical range of 10 points to each bug.
Bug #1: For example, it is a bug that crashes the program (10 points), it exists in a major part of the program (10 points), and it affects 80% of customers (8 points), so this bug causes "user pain" "The volume is 28 points and we bet we're going to fix it.
Bug #2: It's just an arrangement bug (2 points), it appears in the secondary window (2 points), and the part of the program where the bug is located will only be used in older versions (2 points). Therefore, the "user pain" value of this bug is 6 points, and we will probably not fix it.
Unfortunately, many situations are not as simple as the above. Bug #3 is a data loss issue (10 points) that exists in a major part of an application but only fails under certain circumstances (5 points) (the data is subjective, by the way) out). Customer research proves it is rarely used (2 points). Therefore, its "user pain" value is 17 points. This is an ambiguous data, which can be modified or not. On the one hand, the investment required to fix it may not be worth it, but as long as the problem is understood and it doesn't have any blind spots, ignoring the bug is probably the right approach.
On the other hand, you have to weigh it against other bugs in the system. We apply the "Broken Window" effect here - if there are too many such medium-threshold bugs in the application, the quality of the product (or at least, the perception of quality) will be greatly affected. When you consider each bug in the system, you should also consider other (known) bugs in the system, and use this to analyze and decide which bugs need to be fixed and which are not worth fixing.
It is indeed a very bad thing to have bugs in officially released software - but based on our existing development tools and development languages, we have not yet found a more reasonable solution.
Replenish:
As I write this, I think I'm missing a fourth factor in the formula: release date. This factor also plays a key role in the decision to fix/not fix a bug as the release date approaches. However, I'm not sure if it's the fourth factor, nor what the threshold is for the amount of "user pain" required to fix a bug closer to release.