The theme for this week seems to be quality so let’s look at some different aspects of that.
Setting the goal
First of all, let’s consider what is really the goal. We often find ourselves talking about testing rather than quality and that leads us down the wrong path. Testing is not the goal and never should be.
Why would we focus on the wrong thing? It has to do with a psychological phenomenon called attribute substitution, where in an attempt to reduce the mental effort required, we replace a complex decision with a simpler decision and aren’t even aware that we’ve done so.
Quality is a difficult thing to get our heads around so we simplify the problem to testing and focus on that. Yet, most of what we call testing happens too late in the process to actually improve the quality. See this article on quality vs testing for more.
The best we can hope for with typical testing is that we’ll identify when we need to go back and rework the product. If we want the testing to actually improve the quality then it needs to happen during creation of the product, not after the fact. This is the essence of techniques like test driven development where the testing happens before the code is even written.
What we normally call testing is an inspection step, not a quality one, and as Deming says: “Inspection does not improve the quality, nor guarantee quality. Inspection is too late. The quality, good or bad, is already in the product.”
If we want to improve quality then we need to focus on quality. The moment we start talking about testing, we’ve taken our eyes off the ball.
Bugs
If you ask the typical team what a bug is, you won’t be able to get a clear answer. We talk about bugs all the time and yet we all seem to have different ideas of what they are.
I have a clear three point definition for a bug. I don’t insist that anyone uses my definition, but I find that many teams do, because they don’t have their own.
It’s a defect if:
- The team knew what it was supposed to do
- And it doesn’t do that thing
- And the team said they were done
All three are important. If someone insists that it’s broken because it doesn’t do something that we never talked about, it’s not a bug. If we didn’t understand what it was supposed to do, then it’s not a bug when it doesn’t do that thing.
Why it’s important to have a clear definition of that bug, is that every bug is a place where we said we’d finished the work and we didn’t.
Having it so clearly defined will change the whole conversation around prioritizing the fixing of bugs. A bug is something that we’d already prioritized, and already decided to work on but stopped before we were finished. If we’d finished it, there would be no bug.
So now if someone asks if we want to talk about prioritize bugs, it’s really a matter of “do we want to finish this thing that we’d already decided was important enough to have started first, or do we want to do this new feature that was clearly less important because we’re only looking at it now?”
Yes, I realize that priorities change and the newer thing might really be more important, but most of the time that isn’t true.
When you realize that those bugs really are more important than new features, you’ve just taken the first step towards a zero defect policy
You may now be thinking “What if we don’t want to fix it?” Then close the bug. Why are you leaving it in the ticketing system if it isn’t important enough to finish? Yes, it really is that simple.
Have you ever sat in a bug prioritization meeting? You’ll never get that time back and that meeting was 100% waste. Either fix the bugs or decide that you won’t, and cancel them. Trying to prioritize one against the others is a waste of everyone’s time.
That last point in the bug definition of “the team said they were done” has one more interesting implication. We don’t ever open a bug if the story is still showing as open, which means that we can never have two tickets open for the same thing at the same time.
Pull requests
At many companies, Pull Requests (PR’s) are treated as if they were a quality step, and yet in the most common cases, they aren’t. PR’s are an inspection step that happens after some portion of the code is finished.
Can a PR be used to get feedback in the moment, to help with quality as the code is being built? Absolutely yes. Unfortunately, they’re almost never used in that way — instead they’re used for inspection and gate-keeping.
The common case for PR’s is that we create one and send off an asynchronous request for feedback. Then we move on to some other work because we know it will be at least hours, if not days, before we get any feedback. If we were actually using the PR for the purposes of improving quality, we’d want immediate synchronous feedback.
There are certainly people who use PR’s this way. Where they create the PR as a way to see all the changes in one place and then immediately review them with someone else so that they can get feedback that will then inform quality.
The asynchronous model, on the other hand, is a handoff. We’ve said that we’re finished and want someone else to approve it. This is now an inspection step.