Intellectually everyone understands that improving code quality is a good thing. After all, we know bad quality when we see it. (Anyone old enough can cast his or her mind back to the late '80s and Microsoft Word for Windows 1.0.) But we also know that there comes a point where there's a diminishing return on our investment in code quality. How much work would you put into tracking down a bug that's only ever been reported once by a user running on OS/2 Warp?

The problem with code quality initiatives is that we really don't know how much effort to put into them. We have never truly answered the question: how much quality is enough?

Why Code Quality Is Important
The Standish Group famously reports on the software industry's inability to deliver successful projects on a regular basis. In 2004, it reported that just 29% of software projects were considered a "success."

Projects fail for a number of reasons, and as far as the Standish survey is concerned, that means either the projects were completely canceled, or that they were "challenged" because they significantly overran time and/or cost.

There are many reasons that projects are canceled or overrun, and almost certainly the most common is that the software just didn't do what it was supposed to - it didn't conform to requirements. That's a hard problem to fix - and it's an even harder one to measure.

Code quality, on the other hand, which is probably the second largest contributor to project failure, is much easier to measure, and therefore easier to manage. Yet, many development teams waste time on rework cycles fixing bugs that crop up in QA or production that could have been caught and fixed upstream, with relatively little effort.

A big part of the cause of that problem is that during the coding phase, oftentimes the only measurement we have available to track progress is whether we hit the deadline or not. Even if we monitor defect rates (it may shock you to learn that not all development teams do), we just don't have any meaningful metrics by which to manage the process until it's too late.

Agile methods help. Test-driven development, where unit tests are written with, or even before the code, and continuous integration and test cycles flush out many bugs in development before they get to QA.

But fundamentally, managing only to deadlines causes an accumulation of "technical debt" - a term coined by Ward Cunningham to describe the effect of all those decisions to use a quick and dirty approach to solving a problem, with the idea that you'd come back and fix it later. Just like financial debt, technical debt isn't necessarily a bad thing, especially when an important deadline is looming. But like financial debt, if you have too much of it, it can get out of control and becomes very hard to pay back.

How Much Quality Is "Enough"?
Quality is a tradeoff between cost and risk. If you ask most (inter-nal or external) customers what level of quality they'd like to see in their software applications, they'll likely tell you "perfect" - i.e., zero defects. Of course, that's something of a trick question, because you didn't tell them how much "perfect" would cost.

After a 1996 Fast Company article on the Lockheed Martin group that builds and maintains software for the space shuttle program, that software is often cited as the most expensive code on the planet, line for line. I'm not sure anyone really knows the cost per line of the space shuttle software (it's been estimated at $1,000 per line), but we do know that (as of 1996) it took roughly 260 developers to maintain 420,000 lines of code, which comes out at about 1,600 lines per person. That's expensive - but the approach seems to work: according to the article, the previous three versions of the software had only a single defect detected per release.

Of course, a typical business application isn't controlling a space shuttle (which, even in 1996 dollars, cost $4 billion a pop, not to mention the lives of a half-dozen astronauts being at stake), and many development organizations struggle with the cost/risk equation. How much quality is enough for your organization? For your application? How can we even start to answer that question?

Ideally, we should be able to come up with a mathematical solution. In probability terms, the expected cost of failure would be expressed as the cost of something bad happening multiplied by the probability that the bad thing will happen. So, as you spend more on increasing the quality of the software, the probability of failure comes down, reducing the expected cost of failure.

This approach becomes problematic in practice, because it's hard to measure the incremental cost of increasing quality and the probability of failure. But, unless we (as an industry) start to measure these things, we'll remain in the Dark Ages.

The cost to fix a defect depends on many factors. Conventional wisdom states that the earlier in the software lifecycle a defect is found and fixed, the lower the cost. Of course that's true, although depending on the application, it probably costs less to fix an application in production now than it did even a few years ago. (How many times have you opened up an application only to have it ask if you want to automatically download and install the latest and greatest version?)

There are other quality-related metrics that can help get a handle on how well code is being written. Most development organizations have a set of coding standards. How many organizations measure how well the standards are being applied? Coding standards don't just tell you where to put your curly braces. If applied consistently, they can have a real, measurable impact on code quality and maintainability.

There are a number of static code analyzers available that help development teams implement their coding standards. And static code analysis can go further than just checking adherence to standards - it can find bugs that would otherwise slip through the compiler undetected (consider the old chestnut of incorrect use of the "=" operator in a conditional statement).

Unit test activity is another early indicator of software quality. Although it's not perfect, code coverage (the percentage of executable lines of code exercised by your unit test suite) is a good measure of the effectiveness of your unit tests. You have to keep in mind of course that code coverage isn't the same as path coverage (in other words, 100% code coverage doesn't mean that your code is bug free!). But it's still fair to say that, up to a point at least, the higher your code coverage percentage, the better chance you have of squeezing those bugs out.

This is a great example of a place to look closely at return on investment. You should set a realistic goal for code coverage. A development team could spend many hours trying to get the coverage rate from 98% to 100% - with very little return in terms of the number of defects trapped in development. Many of the better-known open source projects show coverage rates in the 50% to 70% range, and I know for a fact that many commercial projects sit at about half that number.

The important point here, to roll out a well-worn adage, is that you can't manage what you don't measure. If you're not measuring defect rates at different stages of the lifecycle, start now. Take a look at some of the tools available for static analysis and code coverage and, more importantly, start to track those quality metrics over time. Only when you have sufficient data to analyze can you start to make some decisions on how far to go with your code quality initiatives, and finally be in a position to answer the question: how good is good enough?