Go here to sign up for The Embedded Muse.
The Embedded Muse Logo The Embedded Muse
 Issue Number 417, March 1, 2021
Copyright 2021 The Ganssle Group

Editor: Jack Ganssle, jack@ganssle.com
   Jack Ganssle, Editor of The Embedded Muse

You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

Contents
Editor's Notes

SEGGER Embedded Studio cross platform IDE

Tip for sending me email: My email filters are super aggressive and I no longer look at the spam mailbox. If you include the phrase "embedded muse" in the subject line your email will wend its weighty way to me.

Congratulations to the Perseverance team for their successful Mars landing. This is a victory for embedded systems, without which the lander would never have had a chance.
Quotes and Thoughts

"The dangers of not thinking clearly are much greater now than ever before. It's not that there's something new in our way of thinking, it's that credulous and confused thinking can be much more lethal in ways it was never before." Carl Sagan
Tools and Tips

Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.

Two good articles about debugging hard faults on Arm processors are here and here. Both of those blogs (from Embedded Artistry and Memfault) are worth following.

And don't miss Memfault's blog about using semihosting on Cortex M processors. This lets your target system use the I/O on your PC via the debugger.
Freebies and Discounts

Courtesy of the folks at MagicDAQ, March's giveaway is one of their Python powered USB DAQ and compatible hardware testing module. 

Muse readers can get a 10% discount (valid till the end of June) on these if they send an email to support@magicdaq.com and mention the Embedded Muse.

Enter via this link.
Getting it Right

How good is your code?

Most of us have no quantitative idea. A few outfits measure bug rates, but, as been often observed, an apparent absence of bugs does not mean the code lacks defects.

While safety-critical code is often better-tested, and sometimes better-designed, than average industrial software, testing cannot prove the absence of bugs. According to Adams, N.E "Optimizing preventive service of software products", IBM Journal of Research and Development, 28(1), p. 2-14, a third of all software faults take more than 5000 execution-years to manifest themselves. That's encouraging for those of us not planning to challenge Methuselah, but graphically illustrates the inadequacy of testing.

Some operating systems are certified to extremely-high levels of reliability and/or security. To my knowledge none have been completely proven using the most rigorous formal methods (some, like Green Hills' Integrity, certified to EAL6+ against a particular protection profile, come close).However, NICTA's Secure Embedded L4 microkernel has been completely proven correct using formal methods. That's quite an achievement.

But more interesting is the stunning cost required to make the proofs. The OS comprises 9,300 lines of code, of which 7,500 have been verified. The cost of verification: 25 to 30 person years of work! At a loaded cost of $200k/person year, that's over $5m, or about $700 per line of code. Since most firmware runs $20-40 per line of code (this covers the entire cost of the project), formal verification is twenty times the cost of writing the stuff in the first place.

Actually, $700/LOC is cheaper than one might think. Green Hills apparently spent about $1000/LOC to achieve their EAL6+ validation, which is not as strong as a proof as what NICTA claims. NICTA themselves believe EAL6 certification would run about $10,000/LOC. That's quite a range; even at the low end it's gobs of cash.

They found 144 bugs while formally proving the OS. That's about a 2% rate, which suggests very little testing was used prior to the proof.

So- how good is your code? How do you ensure it's close to being right?
On Engineering

Steve Johnson had some thoughts about the profession:

I spent much of my career designing the software for medical devices and in the pharmaceutical industry.  The FDA takes a keen interest in that software, as you can imagine, as it has an immediate and potentially life-altering impact.  In doing so, I've observed the following:

Support infrastructure: document management systems (that were, in turn, "Validated" in the FDA sense of the word), defined engineering practices (known and documented expectations for documentation, reviews, sign-offs, scheduled audits, etc.), direct and visible ties between expectations and reviews / compensation / continued employment.

Documentation out the wazoo: conceptual objectives and design, requirements analysis, detailed design, unit test plans, integration test plans, and system test plans.  Not quite the same as Installation Qualification ("IQ"), or what's the expectation and test plan for the system on which the software is to be installed; Operational Qualification ("OQ"), or what's the expectation for the system and staff for the hardware / software system, and Production Qualification ("PQ"), or what is the system expected to do, and does it do those things?  Only when all those (IQ, OQ, PQ) are in place and have been successfully executed and reviewed, is the system considered "Validated" in the FDA sense of the word.

I'd distinguish "art" from "engineering" in the degree to which a particular thing is documented.  One doesn't necessarily expect to see structural engineering done on a picnic table, but the documentation for a suspension bridge is considerable.  I can't find the reference, but I recall a quote: "When the documentation outweighs the plane, the plane can fly."

James Fowkes wrote:

I think one big difference in how our particular brand of engineering differs from others is that it doesn't have to obey physical laws.

I was trying to explain what "software engineering" was to a relative (as opposed to mere "programming"), and stumbled onto the following example: you are building a bridge over a river. You know the weights and strengths of materials, you have weather data for the area, you have knowledge of tides and currents, you have a specification for static and dynamic loads, etc, etc. Since all these components obey physical laws, you can have a high degree of confidence in how they interact, how changes in one affect another, etc. It's complicated, but it's logical.

In software, you simply cannot make these mathematical, grounded-in-physical-law assumptions. A bridge built in software might be fine with a car on it, but fall down when you try to cycle across it. It might be fine if the wind is blowing, but no wind at all makes it shake violently.

I think a great deal of software engineering is mitigating that inherent unpredictability, and making systems as understandable and predictable as possible.
Is the Customer Right?

Vlad Z had a take about my linking to a Karl Wiegers post:

Karl Wiegers, whose books are wonderful, has an interesting post titled 66 Lessons from 50 Years of Software Experience. It's pithy.
"9. Despite what our culture maintains, the customer is not always right."

In my experience, the customer is usually not right.  They often do not know what they want and cannot formulate their problem.  In my experience, you should not listen to the customer.  Instead, you should try to understand what the customer does with the current product and it's deficiencies from the customer's point of view, and help them formulate their problem.  Only then you can offer a solution.

It's much easier with the embedded systems.  Customers are competent more often and are easier to deal with.

"3. Never let your boss or your customer talk you into doing a bad job."
that is a wishful thinking.  I quit 2 jobs because of that: one led me to consulting, the other was during consulting period.  I could afford to do that; many people can't.

My dad always drilled into us that we should always do the best job we can. It took me a long time to realize he was wrong. If I'm woodworking on a yacht everything must fit perfectly, but in constructing a house it really doesn't matter if the studs are off by a few mm. Or even cm. Throwaway test or experimental code often doesn't need the care that needs to go into safety-critical software. So a better moral is: Do the best that the job requires.

And Vlad's comments remind me of a sales guy I worked with while consulting decades ago. He was an ex-engineer, ex apparently because he really wasn't much good at engineering. But we'd visit a customer who would start describing his problem, and before getting halfway through this fellow would jump in with all kinds of technical "solutions." "We'll use so and so CPU, toss in this much memory, ..." Needless to say he closed few deals. And ticked off a lot of people, yours truly included.
A Loaded Bookshelf

Responding to the last issue's suggested book list, Mike Teachman wrote:

You asked:  What books do you consider essential?

"The Pragmatic Programmer: your journey to mastery"

William Grauer suggested a classic:

"The C Programming Language" by Kernighan & Ritchie.  No fluff, just the facts.

And Duane Mattern had a couple of suggestions:

Funny, I was reading "The Art of Electronics" as I received Muse 416.  There are two books that I recommend for software development, both opened my eyes:

 1) Miro Samek, Practical UML Statecharts in C/C++, 2nd Ed
 2) Bowman, Emerson, Darnovsky, The Practical SQL Handbook

#1 completed changed the way that I program.
#2 let me see the relationships between object oriented programming, databases, and spreadsheets.

It's not a book, but for embedded C++, I read Dan Saks old magazine articles.  He dives into the nitty gritty of compilers.  Dated, but still relevant: https://www.dansaks.com/articles.shtml
Failure of the Week

From Neil Peers. Where is Italy?
This Week's Cool Product

Machine learning is all the rage, and Eta Compute is deeply involved. They are bringing ML to the edge, now partnering with Synaptics. Their vision is to support extremely low-power MCU applications, like detecting that X number of people are in a room.

Note: This section is about something I personally find cool, interesting or important and want to pass along to readers. It is not influenced by vendors.
Jobs!

Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.
Joke For The Week

These jokes are archived here.
About The Embedded Muse

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.