The Role of QA

Summary: QA is not supposed to find bugs.

The Quality Assurance team is there to test the system and generate a list of bugs. Developers then fix those, and, after passing QA's tests, the product ships.

Not.

First, in manufacturing QA is not quality control. QA is about defining the processes used, raw material fitness, and the like. Instead, quality control insures that a product meets a standard defined by the QA group.

However, almost every software group conflates QA and QC, generally folding both operations into the single term QA. I have no reason to tilt against windmills so will use the term "QA." Software engineering isn't manufacturing; we don't need to adopt their nomenclature.

I believe that developers have the responsibility to deliver extremely high-quality code. Tiny teams may deliver directly to the customer, while larger groups have a separate QA operation. Regardless, we engineers must create the very best work products.

QA's role is to demonstrate the absence of defects. Sure, life is tough and software complex. Sometimes they will find problems. But that doesn't absolve the engineers of their responsibility to strive for perfection. We engineers must take pride in our work, demonstrate exceptional craftsmanship, and constantly hone our tools, processes and techniques to achieve the highest quality.

Wise managers will use the results of QA's verification and validation work to audit the software engineering process. If they're finding more than a few problems, that's an indication that there's something wrong in the firmware group. This is a quantitative result, something measureable, which can be monitored over time to see if the engineering team is getting better or worse.

Three important principles are embodied in the notion of using QA's results to audit the development team:

 Metrics - Engineering without numbers is not engineering - it's art. Metrics hold us accountable and let us evaluate the process.

 Feedback - Every EE knows that feedback stabilizes electronic systems. It also stabilizes human systems. Use the results of any operation (in this case QA) to continuously improve the process.

 Filters - Software is tough to get right, which means we need to use as many filters as possible. The compiler identifies syntax errors, inspections other problems, test even more. QA is the last one. If the last filter is still finding problems that's an indication something is wrong. or another filter is needed.

QA should also be responsible for constructing and managing regression tests. That should be done outside of the engineering team since it's a meta-operation that requires constant attention, an attention of the sort that is quite orthogonal to writing code. That may mean building and maintaining test jibs, continuous integration servers, and all of the infrastructure verification and validation demand.

Aim for the perfect. When imperfections sneak in, figure out why. Take corrective actions as required.

Published November 5, 2014