You may redistribute this newsletter for noncommercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go to https://www.ganssle.com/tem-subunsub.html or drop Jack an email.

• Editor's Notes
• Better Firmware Faster Seminar
• Quotes and Thoughts
• Tools and Tips
• What I'm Reading
• Compared to What?
• Working Overtime
• Jobs!
• Joke for the Week
• Advertise with us
• About The Embedded Muse

Did you know it IS possible to create accurate schedules? Or that most projects consume 50% of the development time in debug and test, and that it’s not hard to slash that number drastically? Or that we know how to manage the quantitative relationship between complexity and bugs? Learn this and far more at my Better Firmware Faster class, presented at your facility. See https://www.ganssle.com/onsite.htm.

Circuit Cellar interviewed me recently.

No ESC East? No Problem!

What? No Embedded Systems Conference this Fall in Boston?

Instead of spending a week at the cancelled ESC, come to the public version of my one-day Better Firmware Faster class near Boston November 1. It will be held at a hotel near Routes 90 and 495, about 38 miles from Boston's Logan airport and 39 from TF Green (Providence, Rhode Island). This is the first public version of the class in two years.

This is a very fast-paced (and fun!) full-day course that teaches better ways to produce firmware. I'll show how to drastically cut bug rates and shorten schedules. Plus, I cover technical issues that are unique to embedded systems, like dealing with real-time issues.

Did you know that volatile doesn't work reliably in most compilers? Or that the usual fix for reentrancy problems is flawed? How about measuring idle time with a $15 VOM? I'll cover all this and a lot more in this fast-paced class.

There's more information here, and this is the course's brochure.

Seating is very limited so sign up today. There's a discount for registering before October 1.

The sooner you start to code, the longer the program will take. – Roy Carls

Please submit neat ideas or thoughts about tools, techniques and resources you love or hate.

I recently reviewed Saleae's 16 channel USB logic analyzer here.

Apparently we're making too much money. To assure "income equality" (I think they mean equal poverty for all) STEM workers need a serious wage cut.

An over-long and repetitive, but somewhat interesting, take on documentation.

In the incredibly dumb move department, Intel recently demonstrated a processor powered by a glass of wine. Alas, there's no mention of the type of wine used, but this is a resource far too important to squander as a power supply.

Firmware is the most expensive thing in the universe.

In his wonderful book "Augustine’s Laws," Norman Augustine, former Lockheed Martin CEO, tells a revealing story about a problem encountered by the defense community. A high performance fighter aircraft is a delicate balance of conflicting needs: fuel range vs. performance. Speed vs. weight. It seems that by the late 70s fighters were at about as heavy as they’d ever be. Contractors, always pursuing larger profits, looked in vain for something they could add that cost a lot, but which weighed nothing.

The answer: firmware. Infinite cost, zero mass. Avionics now accounts for more than half of a fighter's cost. That’s a chunk of change when you consider the latest American fighter, the F-22, costs a cool third of a billion a pop. Augustine practically chortles with glee when he relates this story.

But why is software so expensive? Tom DeMarco once answered this question with these three words: compared to what? He went on to discuss relatively boring business cases, but that answer has resonated in my mind for years.

Compared to what? With software we routinely create product behaviors of unprecedented complexity. Sure, the code's expensive. But never in the history of civilization has anyone built anything so intricate.

Consider the following bubble sort, lifted shamelessly from Wikipedia and not checked for accuracy:

void bubblesort(int * A, int n){

    for(int i(0); i < n; ++i)
        for(int j(0); j < n - i - 1; ++j)
            if(A[j] > A[j + 1])
                std::swap(A[j], A[j + 1]);
}

It’s a mere 110 non-space characters, perhaps tossed off in an hour or two. Suppose we didn't have software and had to implement a sort using some other strategy. What would it cost?

A mechanical engineer might boast that his profession built sorters long before computers. Consider IBM’s 1949-era model 82 card sorter (http://www.columbia.edu/acis/history/sorter.html) with a throughput of 650 cards per minute, rather less than our code snippet might manage even on a 4 MHz Z80. The model 82, of course, only sorted one column of a card at a time; to completely sort a deck could take dozens of passes.

How long did it take to design and build this beast? Years, no doubt. And its functionality pales compared to our code which is so much faster and which can handle gigantic datasets.

But that was 1949. How long would it take to build a bubble sort from electronic components - without FPGAs and VHDL, or a CPU?

In an hour I managed a rough block diagram, one above the chip level (blocks have names like "adder," "16 bit latch" and the like). But the sequencing logic is clearly pretty messy so I've just tossed in a PLD, assuming at some point it wouldn't be too hard to write the appropriate equations. And, yes, perhaps that breaks the no-programmable-logic rule, but to design and debug all that logic using gates in any reasonable amount of time is as unlikely as buck-a-gallon gas.

Assuming 16 bit words and addresses, the circuit will need around a dozen 16 bit latches, adders, and the like. Plus memory. And I have no idea how the unsorted data arrives into the RAM or how the results get exported. Those are unspecified design requirements.  The software-only solution naturally resolves these requirements just by the act of writing the function prototype.

Translating the rough block diagram to a schematic might take a day. Then there’s the time to design and produce a PCB, order and load parts (and change the design to deal with the unexpected but inevitable end-of-life issues), and then of course make the circuit work. We could be talking weeks of effort and a lot of money for the board, parts and appropriate test equipment.

All this to replace 7 little lines of code. Few real embedded programs are less than 10,000; many exceed a million. How much hardware and how much engineering would be needed to replace a real, super-sized computer program?

Consider a real program like a spreadsheet. How much circuitry would it take to make one without a processor? It could be the size of a city.

Firmware is the most expensive thing in the universe, but only because of the unimaginable complexity of the problems it solves. But it's vastly cheaper than any alternative. So when your boss irritably asks why the software takes so long, you know what to say.

Compared to what?

Intel introduced the 4004, the world's first commercial microprocessor, in 1971. A year later they followed with an 8-bitter.

At the time I was working my way through college as an electronics technician. But the company realized this new technology could enable revolutionary new products. Problem was, none of the engineers knew anything about programming. I did, so was gladly co-opted into the engineering department. It seemed ridiculous that the company was willing to actually pay me to crank 8008 assembly language.

The company was perennially cash-short and just one angry creditor shy of bankruptcy. Engineering was always under pressure to produce some new product that would finally put some financial stability behind 150 people's paychecks. For years several of us young engineers worked insane hours, often putting a hundred, paid for 40. I slept in a VW microbus parked in the lot outside.

The company needed the free labor, but the truth is we reveled in the excitement of the work, and enjoyed some prestige from helping to resolve the company's desperate needs. And, of course, we were young, early 20s, unmarried and for too-long periods not even dating. Who had the time?

The 40-something managers went home at 5. They had wives (there were no female managers there), children, and outside interests.

Now I look back and wonder if this wasn't some rite of passage. Young medical residents live at the hospital for days on end. Raw army recruits spend their days squirming through the mud.

Middle-aged doctors go home at 5. Undeployed Colonels have dinner with the family. Yet engineers routinely work long hours regardless of age.

In the USA most developers are statutorily exempt from the Wage and Hours act. The government doesn't care if we're compensated for overtime, and too many companies happily play along. I see very few that pay time and a half or even straight time for OT. Some offer comp time: take an hour off for each hour over 40 you've worked when the crunch is over. That seems reasonable... though it's rare.

I object to the engineer who's never willing to put in some extra time to save a project in distress, just as I object to the company that demands constant overtime. Compensated or not, I think routinely long hours are akin to indentured servitude.

Perhaps the eXtreme Programming folks advocate one of the most balanced approaches: never work overtime two weeks in a row. They acknowledge that projects do run late, and extra time is needed. But tired people make mistakes, so too much OT turns a troubled project into a disaster.

But should those overtime hours be compensated? In my opinion, yes. Either in money, time, or in stock options.

One study suggests that the size of firmware is doubling every 10 months. Hiring isn't. The implications are clear.

Companies must compensate fairly, or at the very least limit and control the use of overtime.

What's the policy at your company? How do you feel about it?

Let me know if you’re hiring embedded engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intents of this newsletter. Please keep it to 100 words.

Note: These jokes are archived at www.ganssle.com/jokes.htm.

Luca Matteini contributed this. He says it's from an actual data sheet:
 "(5) Do not touch the P.C.B. with bear hands."

Luca suggests avoiding overweight hairy technicians in rough environments.

Advertise in The Embedded Muse! Over 23,000 embedded developers get this twice-monthly publication. .

The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and contributions to me at jack@ganssle.com.

The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get better products to market faster.