Inspections Prior to Test

Summary: Inspections are a cheap way to get good code fast. Do them pre-test.

One of the most powerful tools to improve the quality of firmware is the code inspection. They also speed development. A lot of data supports the latter statement; for example, in "A Survey of Software Inspection Technologies" by Oliver Laitenberger the author shows that JPL's inspections are ten to 34 times cheaper than testing. A wealth of other studies that confirm this.

In my travels this Autumn I've had a chance to visit several companies that do pretty disciplined inspections, which is laudable, and unfortunately uncommon. Interestingly, the companies I've recently visited do the inspection before final commit to the version control system. That is, the developers write and test the code, fix the bugs they find, and then inspect it.

That's a mistake.

Inspections should take place after cleaning up errors found by automated tools, like the compiler, lint, and static analysis. No test should be done pre-inspection.

It's completely understandable that developers wish to test-first. Who wants their mistakes pointed out in a meeting of their colleagues? Sneak in a couple of hours test to get rid of the smack-your-forehead-obvious mistakes.

But inspections are cheaper - much cheaper - than test, so test-first is akin to burning stacks of hundred-dollar bills. It's simply bad business.

A central observation of the quality movement is that you can't test quality into a product. That's also true in software; a focus on fixing bugs will not lead to quality code. Quality must be designed in from the beginning.

When inspections are done pre-test, the code automatically improves. If I'm writing code that no one will look at it's quite different from that which will go through an inspection. And that's part of the goal of the process: encourage the developers to write correct code from the outset. Have them work harder during the coding process, rather than devoting eons of time to fixing all of the bugs - or, at least, all of the ones they can find.

In The Economics of Software Quality, By Capers Jones and Olivier Bonsignour, the authors report that out of 121 known ways to improve software quality, the fifth most effective is pretest inspections.

To repeat: a focus on fixing bugs will not lead to quality code.

We need to break the cycle of writing a lot of code fast and then getting into the tar pit of endless test. Think first. Code carefully. Examine the results. Make it as perfect as possible. Feed it through the tools and fix any errors detected. Inspect it and fix those defects.

Finally, test.

Published October 27, 2014