By Jack Ganssle

Quality is Job One

Published 1/23/2006

In his article "Quality is Now Development Job One" (http://www.eweek.com/article2/0,1895,1896691,00.asp), author Peter Coffee reports that more and more companies are gauging software release dates by quality metrics rather than schedules. Even Microsoft announced that Vista will be held until its quality milestones are met.

Wow.

Yet in the embedded world quality is usually an afterthought. Though I often see projects held up while developers try to deal with a hundred page bug list, it's almost unheard of for management to define quality metrics that will dictate the schedule. The closest too many organizations come to managing quality is to hold a bug deferral meeting long after the deadline went swooshing by to decide the precise level of lousiness they're willing to ship.

It's rare to see management even talk about quantitative quality metrics. Yet we know that it's cheaper to build quality in, rather than attempt to retrofit it at the end.

After all, what's the fastest way to fix a bug?

Don't put it in there in the first place!

That, of course, means we need better approaches than the usual heroics. Standards, inspections, disciplined construction of test suites, complexity analysis and all of the usual well-known techniques are essential ingredients of a project which will delight the customer.

December's Crosstalk (http://www.stsc.hill.af.mil/crosstalk/2005/12/0512Humphrey.html) has an article from the always interesting Watts Humphrey. In "Acquiring Quality Software" he lists six quality principles. The first is that if a customer (or management) does not demand a quality product he or she won't get one.

Well, duh. But saying ". and make it bug-free" is a meaningless statement. Bug-free is not a metric and is not measurable (see Humphrey's paper for more rational metrics). That's especially true with the poor testing endemic to this industry. Lame tests just won't find most bugs so the product appears much better than it is. Admittedly, the IT folks have a much simpler problem then we do, as many embedded systems defy automated overnight testing. Yet crummy unit tests and poorly-thought-out systems checks abound.

Contrast this with the operation of a well-run factory. Graphs posted around the floor show productivity, inventory levels, and quality goals versus actual performance. While it is hard to manage the quality of a creative endeavor like firmware engineering, we must. If we don't, we'll be shipping million line programs harboring tens of thousands of defects.

How does your organization manage and measure firmware quality?