What kind of goals should you have in place to help your team accomplish the things it wants to? How do you measure what you are doing in relation to your goals? If your goal is to have zero defects, the following are some things you should do (or not do) to help track progress towards that goal:
- Ideal–The team learns to author defect-free code and fixes any defects that are found immediately.
- Best–The best-case scenario is for any defect that is found is for it to be fixed right away. What is the time from when a defect is first discovered to when that defect is fixed? Is it a week? A day? An hour? If you can fix a defect in an hour (or even less than an hour), then that defect isn't sitting in a backlog (and wasting the time of your team). Many defects that are found by automated tests can be quickly fixed, whereas defects that are found by customers are often more time consuming (especially when you consider how much time it will take to get the fixed software to them).
- Good–Keep an eye on the leading indicators. There are a few things that can help you understand how well your team is doing towards its goal of zero defects, such as the number of automated tests, the scope of test coverage, and breaking the defects that are found into groups. In general, the more automated tests that you have, the less you have to worry about on a day-to-day basis. This assumes, of course, that the automated tests that your team is writing are useful and are testing the right things. When a well-written automated test fails, it should be easy to understand where in your code you need to look to address that failure. Keep the test coverage bar high, but keep it focused on what you need, or would need. Is there a test for every IF statement? Does your test coverage scale with your code as it changes and grows? When defects are tracked in groups, you can use the number of defects as an indicator to refactor a piece of code and to bring it up to current standards or when to replace that code with something that can provide more value to your customers. Also, when your testing effort is broken down into smaller areas, it will be easier to identify how to make a fix.
- Poor–Watch the trailing indicators, but do not spend too much time on them. There are some things that are easy to track, but do more to identify the symptom than the problem. For example, tracking defects found by individuals or tracking the number of defects by priority. These are things that show what you have done or what someone has done. They don't show what you are doing and are, at best, an interesting place to start when it comes to defect tracking and management.