You may redistribute this newsletter for noncommercial purposes. For commercial use contact jack@ganssle.com.

• Editor's Notes
• Quotes and Thoughts
• Tools and Tips
• Bug Bounties
• How Much Effort Should Go Into Architecting a Project?
• Walter Mitty Dreams
• Jobs!
• Joke for the Week
• Advertise with us
• About The Embedded Muse

Court testimony about the recent Toyota ruling makes for interesting - and depressing - reading. A lot of code was involved and some of it was safety critical. Yet it seems the firmware was poorly engineered. No doubt the typical mad rush to get to market meant shortcuts that, at the time, were probably seen to save money. A billion dollars later (with many cases still pending) that judgment looks more foolish than sage.

After over 40 years in this field I've learned that "shortcuts make for long delays" (an aphorism attributed to J.R.R Tolkien). The data is stark: doing software right means fewer bugs and earlier deliveries. Adopt best practices and your code will be better and cheaper. This is the entire thesis of the quality movement, which revolutionized manufacturing but has somehow largely missed software engineering. Studies have even shown that safety-critical code need be no more expensive than the usual stuff if the right processes are followed.

This is what my one-day Better Firmware Faster seminar is all about: giving your team the tools they need to operate at a measurably world-class level, producing code with far fewer bugs in less time. It's fast-paced, fun, and uniquely covers the issues faced by embedded developers. Information here shows how your team can benefit by having this seminar presented at your facility.

"In any electrical circuit, appliances and wiring will burn out to protect fuses." - Robert Byrnes

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

A lot of vendors ask for me to promote their products in the Muse. My policy is to only give attention to products I review or otherwise feel strongly will benefit engineers. Well, Saelig (full disclosure: they advertise on my web site) has a monthly newsletter which in April is promoting a two channel, 50 MHz bench oscilloscope for $259 if you use discount code "S1504C". And there's free shipping. It seems like a terrific value. I ordered one (and am paying for it personally) to review. After that I plan to give it to a lucky Muse reader. This is a GW Instek GDS-10520U. I've never used a product from GW Instek, but they are one of a number of Chinese companies making inexpensive test equipment. I reviewed a Siglent scope, also made in China, here, and think it's a great scope for a really decent price. Now, none of these are Agilent (now Keysight Technologies) or Tek quality scopes, but are at a price point that is affordable for even a home lab. The Saelig promotion is for April only, which is why I'm mentioning it now. If anyone is using this scope now, please let me know what you think of it and I'll pass your comments on in this newsletter.

Phil Martel has another resource for approximations:

Tyler Herring sent a link to a great article on static analysis.

A couple of readers have recently asked about requirements tracking tools. What do you folks use?

This month's (April 2015) issued of the Communications of the ACM has an interesting article called "Privacy and Security, Toward More Secure Software." Dorothy Denning explores two options for dealing with vulnerabilities. The first, which strikes me as appalling, is to have the Federal government buy exploits. They'd pay a bounty for each bug discovered; she suggests up to $100k/bug. Given that NIST added about 8000 vulnerabilities to the National Vulnerability Database last year, that's the better part of a billion dollars per year. Ignoring the cost, and the question of whether this is an appropriate function of government, this strikes me as a bad idea. It will probably incentivize developers to seed their code with defects. And we know the Feds are already buying exploits, but hoard them for their own use, so it's likely multiple arms of the bureaucracy will, as usual, be at loggerheads.

The second option she proposes is to hold software companies liable for damages resulting from attacks. Today, most vendors of non-embedded code are protected by their license agreements. In the US we can sue anyone over anything so one wonders how likely that idea is to hold up in court. In the embedded world these "agreements" are probably toothless since the buyer never signs anything, or even clicks on an "I agree" button. I'm not a lawyer but find it hard to imagine that an agreement has much worth if it hasn't been agreed to by both parties.

But the law is a strange arena. In 2003 attorney Cem Kaner said "Unless someone is injured or dies, it is almost impossible to successfully sue a software publisher for defective software. The serious proposals to change software law have primarily been to reduce software vendors' liability even further. The most recent battles involve embedded software. You might soon discover that when you buy a car, the body is covered by one set of laws but the software that controls your brakes, fuel injectors, etc., is covered by a different set of laws that are more manufacturer friendly."

In The Economics of Software Quality Capers Jones showed that low-quality software ships with about 5.8 times as many high-risk defects as does high-quality code; the implication is that we do know how to achieve high quality levels, given sufficient motivation. And high-quality levels do not necessarily correlate with expensive code. The SPARK people produce incredible software for less than the usual heroics. Other studies, for example Safety Critical Software and Development Productivity by O. Benediktsson, demonstrate that following a reasonable development process can yield best-in-class code for no more than the usual bug-ridden stuff.

Litigation might eventually cause companies to rethink their development processes, but that is always ex post facto. It usually doesn't happen, and when it does, occurs long after the development effort that created the problems. Few managers will see it as a threat to today's engineering efforts.

There has been a lot of recent press about buggy code causing large financial losses. Perhaps investors will eventually demand that publicly-traded companies augment the Risks section of their annual reports with another caveat: "Another potential risk area is that we have made the decision to use poor software development processes, and so are at risk of missing market windows, crippling litigation and enormous fines."

I'm often asked what percentage of the engineering effort should be spent on architecture and design. It's probably not a meaningful question as different sorts of projects will have different answers. But NASA has studied this and for flight software, using the well-known and hardly-used COCOMO-II model, came up with this fascinating graph:

The little circles are "sweet spots" where the overall effort is minimized. Clearly, the bigger the application (in SLOC, or source lines of code), the more effort should go into up-front architecting and design. Of course, this information is hard to use since, at the outset of a development effort, it's hard to impossible to assess how many lines of code will be involved; the earliest time any size information is available is after architecting the system, so the graph is useful more as guestimation than as something for quantitative predictions. But it's useful, and unsurprising, to note that huge projects need more up-front work than tiny ones.

"Architecture" is usually poorly defined. That same document uses it in this fashion: "The software architecture of a program or computing system is the structure or structures of the system, which comprise software elements, the externally visible properties of those elements, and the relationships among them."

An hour into the Hong Kong-bound flight an attendant came on the PA and apologized for the broken video system. "We're calling our maintenance people and will work with them," she assured us.

As a confirmed nerd long cloistered in various labs my life is short on action stories. I have no tales of heroism under fire, nor have I performed an emergency tracheotomy with a pocket knife by flashlight. The President doesn't call when he needs urgent advice on complex foreign policy matters.

But I do know about embedded systems, and here was my chance to live a Walter Mitty dream! No, the aircraft wasn't in danger and there were no lives at stake, but 300 fuming passengers were facing 16 hours crammed into the A340 with the prospect of no entertainment. Things could have gotten ugly. Now an engineer could save the day.

But they'd probably never do a Hollywood dramatization ("based on a real story!") of the rescue.

The lead flight attendant listened politely till she understood that I wanted her to cycle the system's circuit breaker. Then she backed away and lost her plastered-on "can I help you?" smile and firmly suggested I return to my seat. Her arched eyebrows and now severe expression made it clear that just offering advice about messing with any part of the aircraft's electronics raises the terrorist specter. Fine; I had what turned out to be a beautifully-written 800 page book (The Way the Crow Flies by Ann-Marie MacDonald) to while away the hours.

Half an hour later maintenance apparently came up with a solution. The same attendant announced they were going to (surprise) turn the system off and then back on. The screens went blank for a moment till a penguin appeared which like a Buddha benignly presided over a command-line interface. Several hundred Bash-shell commands screamed off the displays. Twenty minutes later, the boot complete, passengers had their video back.

On the return flight we were again treated to a mid-ocean reboot of the same system, though this time I kept my mouth shut. My dream of leaping into the fray to save the day -- even if that meant nothing more than doing a routine reboot -- remains as much a fantasy as Walter Mitty's reveries.

Engineers rarely dramatically save the day. In real life it's Delta Force; in the movies count on Bruce Willis.

There still are those daydreams when I think that the bleating phone will be a summons to an emergency meeting with President Obama. But he'll probably just be looking for advice about rebooting his iPod.

(Walter Mitty was the protagonist of a short story by James Thurber. It runs about 4 pages. I recently saw the 2013 film, and was surprised by two things: the movie bears essentially no resemblance to Thurber's story, and runs about 10 times as long as it takes to read the original).

Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intents of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

Note: These jokes are archived at www.ganssle.com/jokes.htm.

Buddy Smith sent this link about email only working within 500 miles of the sender.

And Dejan Durdenic sent in this update about dark-emitting diodes.

Advertise in The Embedded Muse! Over 23,000 embedded developers get this twice-monthly publication. .

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.