Home
  | 0 - 1 |  
Nat Russo [userpic]

Quality Assurance does NOT begin with the QA Team.

July 30th, 2008 (09:44 am)
contemplative

current mood: contemplative

In my experience in software engineering, I've found that many organizations claim to be "quality-centric", or may have even achieved a certain CMM level.  In reality, however, these organizations rarely QA their products beyond simple manual test cases designed and implemented by a QA team (when the organization in question even has the luxury of a QA team).  These organizations believe that unit tests (generated by Visual Studio), combined with a manual test process, will shake any existing bugs out of a system.  They are correct in thinking they will find bugs in the product, but not for the reasons they think they are correct.

Most organizations wait until the end of a development cycle to begin thinking about QA.  They schedule a certain period of time at the end of the cycle to test and debug.  Yes, they are going to find bugs.  Moreover, the fact that they will find and fix bugs during that "phase" of development will have the overall effect of convincing that organization of the correctness of their approach.  "See!  We TOLD you we'd find bugs!"  These organizations tend to suffer from one of the "Self-fulfilling prophecies" mentioned by Steve McConnell in his book Code Complete ("We'd better start coding right away because we're going to have a lot of debugging to do.").  Incidentally, if you don't have a copy of that book yet, run to the nearest book store and buy it.  Anyone who calls themselves software engineers today should have a worn-out copy of that book on their desk.  But I digress....

The primary function of quality assurance is to avoid introducing bugs to begin with!  In the event that bugs are introduced into the product, then the secondary function becomes detecting and fixing that bug as close as possible to the time it was introduced.  It is possible, not to mention highly desirable, to produce defect-free code.  Don't let anyone tell you otherwise!  No human endeavor is perfect, but that fact is often abused by developers who are, quite simply, too lazy (or burned out) to implement quality processes in their day-to-day routine.  So, how do we avoid introducing bugs to begin with?

The first step on this path is recognizing that quality assurance does not begin with the quality assurance team!  It begins with the individual developer at the keyboard who is trying to decide whether or not to expose a certain method on a class, or whether to refactor a certain block of code into a method of its own, or whether or not the class itself exposes a consistent level of abstraction.  (We could easily get derailed here by discussing upstream activities.  And while I agree that much of the individual developer's success is determined prior to his even being hired, I'm going to take an "all else being equal" approach to this subject and concentrate exclusively on the individual engineer).

We've set the stage, so now let's talk about some specific things a developer can do in his/her day-to-day routine to increase the quality of their product.  Here are a few of my favorites, in no specific order...

Agree on a set of coding standards early in the process
At the end of the day, it doesn't matter whether you use Hungarian Notation, or decide you're going to preface all of your Int32 variables with "George_", as long as your entire team/organization agrees on some standard and you document that standard.  Discussion about coding standards can quickly devolve into mini holy wars, so I'll try to avoid that.  Suffice it to say that the purpose of a coding standard (within the context of Quality Assurance) has little to do with the quality of the standard itself.  The purpose of the coding standard is to ensure that all developers, both present and future, understand how to interpret code they have before them.  Code that follows a strict set of conventions goes a long way towards achieving that lofty goal of self-documentation.

Review all code during development
Many technical leads are reduced to nothing more than code "traffic cops" in certain organizations.  They go through a typical day from one code review meeting to another code review meeting, peering at sample code and trying to keep the development teams on the proper coding standard path.  This is a very limited view of code review, and it turns code review sessions into interrogations where developers feel they must defend every underscore and semicolon.  That is not the sort of "defensive coding" we want our junior developers to employ!  Code review serves three primary purposes:

  1. Catch poor design implementation before other coding efforts become dependent on the poor implementation.
  2. Enforce proper coding standards and naming conventions, etc.
  3. Educate all developers (not just the junior developers).
Many organizations I've been associated with focus so much on item #2 above during review sessions, that they miss fundamental implementation problems, and they miss countless opportunities to educate their developers.  Well-planned and properly-executed code review sessions will have a dramatic impact on the level of quality in your product.  In fact, well-executed code review sessions are so beneficial that the impact is not solely experienced by the current project!  Well-executed code review sessions increase the overall body of knowledge of your fellow developers (team leads included), which has a beneficial impact on any future projects your team tackles.

TDD:  Test Driven Development
There was a time when writing tests was exclusively the responsibility of the QA team.  But it has been shown, time after time, that developers who are savvy about test development routinely produce a higher quality product.  Before I talk about what Test Driven Development is, let me address what Test Driven Development is not

It is not Tester Driven Development.  It is not the purpose of the test team to decide what should or should not be developed.  Those decisions should flow naturally from the body of project requirements.

Test driven development begins by developing a test.  That sounds a little counter-intuitive, until we dig a little further under the hood.  By developing your unit test first, you are making a statement about the class you have yet to implement.  You are defining the context within which "success" can be claimed.  In other words, you are explicitly stating the conditions under which a particular class can be considered "successfully implemented".  The first time you compile this code, the test should fail...because there should be no class implementation yet.  The theory is that you develop the class until such time as the test passes.  When the test passes, you are done.  That class is now feature complete, and you can move on to other classes.

This process is valuable for a number of reasons:
  1. It forces the developer to understand the class before he/she implements the class!
  2. It catches most non-integration defects at the time they are introduced, which is the optimum time to catch any defect.
  3. It avoids waste by letting the developer know when they are "finished".
Number 3 above is very important.  I've lost count of the number of times I've seen a good implementation go bad because the developer didn't know when to quit.  Being a perfectionist comes along with the territory of being a software developer.  Developers, by nature, tend to have a higher attention-to-detail than most other people.  But this can become a liability if it isn't kept in check.  We often desire to implement "enhancements" (or whatever we want to call them) in an effort to make our class perfect.  We tend to lose sight of the fact that perfection is defined by our requirements, not our personal goals.

"Stop the Line!"
I've recently been reading a book named Implementing Lean Software Development From Concept to Cash by Mary Poppendieck and Tom Poppendieck, at the behest of my employer.  This is a very interesting book for any software engineer who is interested in implementing more efficient processes that have withstood the test of time.  The book documents the amazingly efficient manufacturing processes of Toyota, and how those processes can apply directly to software development.  One of the principles (amongst many) the book recommends is taking what the authors call a "Stop the line" approach to development.

At Toyota, the company has given each individual employee who is involved in the manufacturing process the ability to stop the production line at the first sign of a defect.  The line comes to a complete halt, and no further production takes place until the cause of the defect is discovered, and a solution is put in place.  This one seemingly-obvious concept has dramatically reduced the number of defects that pass through the quality assurance process and ultimately end up in the hands of consumers.

At Thermo Fisher Scientific, we have translated this process into our own iteration-based development cycles.  When a defect is detected, development stops, and we find the cause of the defect before moving on to implementing more features.  We have gone so far as to declare a build "broken" if any single one of our hundrends of unit tests fail.  What this means for us is that we cannot consider a feature implemented if it contains a defect of any kind.  The cost of moving forward will simply be too great if we hold off on fixing our defects until some future point in time.

Manage Complexity
Steve McConnell, in his book Code Complete, reiterates several times (himself referencing an earlier author, whose name and work eludes me at the moment) that the prime directive of software engineering can be summed up in two words:  manage complexity.  The way in which we make a quality product is to implement it in the simplest fashion possible.  One of my pet peeves as a software engineer rears its ugly head whenever I see code that appears to be complex for complexity's sake.  There was a time when it was laudable to be as cryptic as you possibly could be.  He who fit as many operations as possible into a single line of C/C++ code was the winner.

Then we, as an industry, started adding up the costs associated with software development and came to realize the folly involved in that approach.  Obfuscation is a noble effort when the goal in mind is security or protection of a valuable intellectual property.  But we should never obfuscate something that is intended for consumption by other developers!  We obfuscate our code in subtle ways that fall far short of direct encryption, but nevertheless have a similar impact.  We obfuscate our code every time we use a "magic number", or create an inheritence hierarchy that resembles a pictogram of the Tree of Life.  We obfuscate our code whenever we create a series of sequential operations, but fail to document the sequence.  We obfuscate our code whenever we see that a method has a cyclomatic complexity measured in 4 digits, but fail to perform even the most basic refactoring.  We obfuscate our code when we insist on abbreviating all of our variables, even though the abbreviation only makes sense to us.  And the list goes on....

The list above represents just a small fraction of the things we can focus on to increase the level of quality in our product.  What I find interesting is that the guidelines for increasing product quality also seem to be common-sense guidelines to becoming better software engineers.

  | 0 - 1 |