You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email.

• Editor's Notes
• Quotes and Thoughts
• Tools and Tips
• Freebies and Discounts
• Built-in Self Tests
• Interrupting Interruptions
• NRE v. Recurring Costs
• Failure of the Week
• Jobs!
• Joke for the Week
• About The Embedded Muse

Tip for sending me email: My email filters are super aggressive and I no longer look at the spam mailbox. If you include the phrase "embedded muse" in the subject line your email will wend its weighty way to me.

The Embedded Online Conference will take place next month. I'm giving a talk. The cost is $190, but if you register here with the promo code GANSSLE the cost is only $90. This event was a huge success last year.

In the last issue a reader mention Tracy Kidder's book The Soul of a New Machine, which is one of the best tomes about engineers at work. In this case they were building Data General's next great computer. Grant Beattie sent this article which is a follow up to the book twenty years later. A fun read.

Steve Leibson wrote a history of the 8008, the first commercial 8 bit MCU. It's an interesting read.

Responding to the last issue's thoughts on abstractions, two readers sent in appropriate quotes.

From John Grant:

John Hartman wrote:

Do you use PTC Axeda agent or Axeda Desktop Server? These are in hundreds of IoT devices. Recently some severe security vulnerabilities have been discovered. More info here. Incredibly, one of the problems is a hard-coded logon credential. A list of known devices with this code is here. Strangely the list includes NXP's LPC1768, which is just an MCU.

Using embedded Linux? Better check out CVE-2022-0847 which can give any user root privledge.

This month's giveaway is a copy of The Embedded Systems Dictionary, signed by yours truly.

Enter via this link.

Per Söderstam has some intriguing thoughts on BIST:

I think there's another aspect to BIST: the datasheet. All too often marketing demands some unexplained form of BIST so it can appear on the product's datasheet. The tests may mean nothing, but are seen as yet another selling point.

When the first IBM PC came out the manual included the BIOS's source code. It started by testing the functionality of each machine instruction, halting if a failure was found. The logic of this baffled me: if instructions were to fail the code would halt or go off into the weeds. Either way the result would be a hung machine. The tests proved nothing.

Today I see plenty of products that run RAM tests. But a RAM failure, especially in an MCU, will almost certainly be massive. A push/pop or call/return in the code that reports the problem will fail. The implication is that these tests have to take place practically at the startup vector, be written in assembly, and avoid all RAM use.

In some instances it makes sense to look for a single event upset, such as from a cosmic ray, or to check the integrity of a data structure on boot. But your run-of-the-mill RAM test is generally pretty silly.

(And most of these are poorly implemented. The common 0x55, 0xaa test simply does not work. See this for details on creating robust tests).

Have you been ordered to include tests that make no sense?

For my money one of the most important books about software productivity is DeMarco and Lister's Peopleware. For a decade the authors conducted "coding wars" at a number of different companies, pitting teams against each other on a standard set of software problems. The results showed that, using any measure of performance (speed, defects, etc.) the average of those in the 1st quartile outperformed the average in the 4th quartile by nearly a factor of 3.

Surprisingly, none of the factors you'd expect to matter correlated to the best and worst performers. Even experience mattered little, as long as the programmers had been working for at least 6 months.

They did find a very strong correlation between the office environment and team performance. Needless interruptions yielded poor performance. The best teams had private (read "quiet") offices and phones with "off" switches. Their study suggests that quiet time saves vast amounts of money.

Think about this. The almost minor tweak of getting some quiet time can, according to their data, multiply your productivity by 3x! That's an astonishing result. For the same salary your boss pays you now, he'd get essentially 3 of you.

Too many of us work in a sea of cubicles, despite the clear showing how ineffective they are. It's bad enough that there's no door and no privacy. Worse is when we're subjected to the phone calls of all of our neighbors. We hear the whispered agony as the poor sod in the cube next door tries to work it out with his spouse. We try to focus on our work... but being human the pathos of the drama grabs our attention till we're straining to hear the latest development. Is this an efficient use of an expensive person's time?

Later studies by other researchers found that after an interruption it takes 15 minutes to get into a state of "flow," that Spock-like trance where you're one with the computer. Yet the average developer gets interrupted every 11 minutes.

Yet the cube police will rarely listen to data and reason. They've invested in the cubes, and they've made a decision, By God! The cubicles are here to stay!

This is a case where we can only wage a defensive action. Educate your boss but resign yourself to failure. In the meantime, take some action to minimize the downside of the environment. Here are a few ideas:

• Wear headphones and listen to music to drown out the divorce saga next door.
• Turn the phone off. If it has no "off" switch, unplug the damn thing. In desperate situations attack the wire with a pair of wire cutters. Remember that a phone is a bell that anyone in the world can ring to bring you running. Conquer this madness for your most productive hours.
• Know your most productive hours. I work best before lunch; that's when I schedule all of my creative work, all of the hard stuff. I leave the afternoons free for low-IQ activities like meetings, phone calls, and paperwork.
• Disable the email. It's worse than the phone. Your two hundred closest friends who send the joke of the day are surely a delight, but if you respond to the email reader's "bing" you're little more than one of NASA's monkeys pressing a button to get a banana.
• Put a curtain across the opening to simulate a poor man's door. Be sure  others understand that when it's closed you are not willing to hear from anyone unless it's an emergency.

The ultimate irony of cubicles is that shortly before he died in 2000, Robert Propst railed against cubes, calling them "monolithic insanity."

Robert Propst invented the Action Office, which was eventually perverted into the cubicle.

Sometimes we engineers forget that engineering is a business endeavor. Our role is simple: help the company make a profit. Everything else is secondary. Without profits a company fails. With them it prospers.

It's critical we make engineering tradeoffs that facilitate that profit. Non-Recurring Engineering (NRE) costs (the price of developing a product) has to be balanced against other factors, like the cost to manufacture the widget. Saving $50k of engineering effort by adding $5 of parts in a system whose product lifecycle won't exceed, say, 1000 units, is often a sound engineering and business decision. Conversely, it can make sense to add a lot of effort to removing a few bucks from a high-volume product.

These judgments cannot be made without a lot of information about costs and volumes, which too few of us are privy to. This means we're operating in an information vacuum which makes it impossible to make the best decisions for the company. It's like driving with your eyes closed. Sure, you might get there, but probably won't.

A correspondent who wishes to remain anonymous sent an email recently which sparked my interest. He wrote: "Our finance department recently did us a great service by identifying the breakeven point in a development cost vs. product cost tradeoff analysis. This breakeven took into account lost opportunity cost in the higher development cost scenario, lost profit in the higher product cost scenario, gross vs. net revenue, and profit targets. The result was a guideline that our teams can tailor to their program (since the tradeoff is dependent on a particular products lifetime sales volumes)."

Wow! This is a highly disciplined outfit which has crossed traditional departmental barriers to give the engineers the information they need to build products optimized for feeding the bottom line.

You'd think more organizations would adopt such a gestalt approach to profits. Yet I suspect few do. By walling this information off from those who need it most the odds of creating a successful product or company wanes.

Instead, all we hear are the screams about schedules.

What about your company? Do you get these sorts of briefings?

From Alan Altman:

From Anthony:

Have you submitted a Failure of the Week? I'm getting a ton of these and yours was added to the queue.

Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter. Please keep it to 100 words. There is no charge for a job ad.

These jokes are archived here.

From Clive Souter:

The person who invented predictive typing has died.

May he restaurant in peace.

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.