|
You may redistribute this newsletter for non-commercial purposes. For commercial use contact jack@ganssle.com. To subscribe or unsubscribe go here or drop Jack an email. |
| Contents |
|
| Editor's Notes |
Tip for sending me email: My email filters are super aggressive and I no longer look at the spam mailbox. If you include the phrase "embedded muse" in the subject line your email will wend its weighty way to me.
|
| Quotes and Thoughts |
Ignorance more frequently begets confidence than does knowledge; it is those who know little, and not those who know much, who so positively assert that this or that problem will never be solved by science. Charles Darwin |
| Tools and Tips |
Please submit clever ideas or thoughts about tools, techniques and resources you love or hate. Here are the tool reviews submitted in the past.
Get confused by all of the USB variants? Here's a useful cheat sheet. |
| Freebies and Discounts |
This month's giveaway is a STM32F411RE, mbed-Enabled Development Nucleo-64 series ARM® Cortex®-M4 MCU 32-Bit Embedded Evaluation Board. As it has a debug port, it's perfect for learning about working with ARM Cortex M4 MCUs.
Enter via this link. |
| On Supply Chain Woes |
A lot of readers have been corresponding about their problems with getting parts. It seems an awful lot of engineering time is being devoted wasted in redesigning boards just to use in-stock components.
Murdo McLeod wrote:
|
With reference to the discussion on replacing microcontrollers and microprocessors, I have noticed a trend amongst operations departments (ie production and procurement) to push for Engineering to replace micros with alternatives, especially in any new designs.
The main reason being it covers up their mistakes in not placing orders early enough, ignoring warnings from the Engineering department and suppliers, and not planning for new projects which required procurement in advance of completion.
"Engineering had to redesign the new board and rewrite the firmware because they chose chips we couldn't buy" is a lot better than "we were given advance BoMs for the initial production runs but didn't buy anything till it was too late and now the project is a failure on the timelines."
Legacy products get aren't under such pressure because production can simply blame the supply crisis without admitting they ignored warnings and didn't prepare properly by bringing order dates forward to meet leadtimes.
Not that I would base my suspicions on real life examples you understand…
The problem I see with this approach, especially for new designs, is quite simply everything is on a 52 week leadtime. So selecting a new micro, redesigning the PCB, doing a prototype build, rewriting and re-qualifying the firmware, all adds a year (eg) to the project timescale and you're then at the back of a 52 week queue for the new micro anyway. But I guess that is better than production admitting or facing up to their screwups.
Nothing is free in Engineering and as Gerald Weinberg said "No matter how it looks at first, it's always a people problem."
|
Claus Knudsen has an interesting solution that will make PCB vendors happy:
|
We are using an FPGA on a couple of our high tech. low volume boards. That particular FPGA is usually around $50/pcs and is now 'on sale' for $2500 to $3000. And with a lead time > 60wks!!
My solution was to redesign the PCBs with an old FPGA, not recommended for new designs, but available @ ~$5/pcs. We bought all 500 we could find AFTER I managed to move the design from the current FPGA to this new (old) FPGA. Since the PCBs are low volume, we bought time (around one year) to think of a more permanent solution.
I made the new design into modules: The main DSP board and a stamp size FPGA module. So next time we may have a need for another FPGA, just change the module. Saves time and CE compliance testing. |
Ian Cull wrote:
|
Some comments on unobtainable microcontrollers (and other parts!)
My company has over the last five years been working to move all products from a circa-2003 microcontroller, to more modern ARM-based ones; the newer parts are less expensive, have more powerful embedded capabilities (multiple fast ADCs for example), and better development tools. But those parts have largely become unobtainable!
In some cases we've been able to purchase a bulk of similar parts, for example the same microcontroller in a different package; or a similar part with not-important differences (maybe just one ADC, which is ok for some of our designs). In one instance we purchased a bulk of the old microcontroller, and redesigned the new product to use that old part!
In one case we completely redesigned a (smaller) product to use an Atmel part that showed more than 250K stock; by the time three weeks later when we proved the redesign, stock was already halved.
In one instance, we found 50K obsolete parts from a manufacturer we'd never used, but the parts appeared to be capable of doing our design; we are midway right now through learning this new-to-us part and implementing our design using it. The biggest challenge has been getting hold of design documentation (because it's obsolete) and program/debug tools.
But it's not just microcontrollers; we heavily dependent on particular automotive highside drivers and they have become very unavailable - we've been forced to redesign a few products to use individual transistors and incorporate discrete overcurrent shutdown circuitry!
Our overall response so far is to be flexible; we keep watching for stock of anything that could be useful for a design, and we're ready to rapidly review pincount / functionality / supplies / etc to decide if we think we can use it - if we can, it's a very rapid design/develop cycle to try and get a few PCBs running and proving the design within 2-3 weeks, so we can purchase a quantity while it's still available.
And a warning: we did purchase a big batch of those troublesome highside drivers from one "dubious supplier", to discover they were non-functional and worse than scrap! |
Elizabeth Russell is dealing with a 37 year lead time:
|
In regards to product lead times, I have been affected mostly by unavailable processors and battery management components. I have needed to redesign some parts of the circuit every time boards are ordered. Sometimes this is just a footprint change but often times requires code changes as well. I would also like to hear about others' experiences with dubious distributors. I have been hesitant but would not want to miss an opportunity to get the components I need.
I have attached a screen shot of a product delivery time for an eCompass chip. I don't know whether this should be included as a failure or just an example of supply chain problems. Some of your readers may be able to take advantage of this timing but I know I won't be around!
|
|
| Thoughts on Metrics |
A few comments on metrics:
First, engineering without numbers is not engineering. It's art. We measure our system's response time, sometimes to sub-microsecond levels. The current consumption. Power dissipation. We should also be collecting metrics about our development processes.
Yet metrics can be perverted. A classic Dilbert cartoon shows the pointy-haired boss offering $10 for every bug the team finds and fixes. They jump for joy! Riches are at hand! Wally will code himself a new minivan that very afternoon! For the wrong incentive can lead to bad results.
If your boss thrives on silly metrics I suppose you could do worse than the first derivative of the Friday beer tab to measure burnout. Or measure the rate of consumption of a centrally-located bottle of aspirin.
It seems to me we use measurements for three reasons:
- To motivate. Yet that simply does not work. The great quality guru Deming studied motivation and found there are two kinds: extrinsic and intrinsic. The first are those imposed from without: the boss says "do this silly thing." We reply that, alas, that is indeed a bad idea but are countered with "do it anyway." Our only-too-natural response is a despairing "fine, but the product is going to suck." By contrast, intrinsic motivations come from within. Professionalism. Being part of a great team. Taking pride in our work. Deming showed, unsurprisingly, that extrinsic motivators drive out the intrinsic. While that might be fine for an assembly line, it's deadly for creative work like engineering.
- To reward and punish people. As a young engineer my boss told me that if I finished a product on time, I could go with the product to the show in Germany. I got my European vacation... but the product didn't work all that well. Like the story with Wally above, we're good at optimizing our own outcomes.
- To promote insight and understanding. To me this is the only reason to measure our engineering efforts. A mantra of the quality movement is Continuous Improvement, which has given us cars of unprecedented reliability. We need the same effort in our work.
I think there are three qualities of useful measurements:
- They must be easy to do. No one will conform honestly with onerous burdens.
- They must give us useful insight into the processes we use.
- They must support effective change-making.
The last cannot be stressed enough. I did a survey of this some time ago and only about 5% of respondents claimed they routinely measure aspects of their engineering processes. A followup question gave a more disturbing answer: Of those that do measure things, 80% do nothing with the data. Time is too short to do ineffective things. If you won't use the metrics to change the process, don't collect the numbers. |
| Determinism |
Engineering is all about building predictably reliable systems. But most firmware engineers ignore the role of determinism in real-time systems. Few can answer questions like "how can you guarantee that the system won't fail when stressed?"
Today's hardware is often cursed with all sorts of nifty speed-enhancers like cache, pipelines, and speculative execution. All of these contribute to execution time uncertainty. The system's performance can vary wildly depending on a lot of hard-to-predict events. An interrupt may occur at any time, and will require at least a partial cache flush. Resuming execution flow means rereading instructions from L2 or memory, which can take a surprisingly long time. Using SDRAM? That initial fetch will stall the machine.
A system that is running fine but close to the edge may suddenly crumble in meeting its hard real-time deadlines.
Can you really guarantee the highest priority task will complete on time? What if there's a perfect storm of interrupts? Or of bus activity (DMA or having to yield the bus to another master)? In big systems a task may depend in very complex ways on externalities (other computers, systems, I/O) that aren't ready in time.
Preemptive multitasking is itself inherently non-deterministic, though techniques like rate-monotonic analysis can mitigate the problem. But RMA requires more analysis than most developers will ever do. In the real world, RMA will likely not deliver useful results.
Even extremely simple systems that have none of these speed-enhancing features can suffer from serious timing problems. A little bit of C code that looks quite deterministic probably makes calls to the black hole that is the runtime library, which is generally uncharacterized (in the time domain) by the vendor. Does that call take a microsecond or a week? No one knows.
It's my belief that too many systems "work" due only to divine intervention. Developers chase down the usual procedural bugs and then breathe a sigh of relief that, once again, a miracle has occurred. But all too often that gift from heaven is merely a reprieve, an indulgence, with damnation still possible or even likely when the system experiences unexpected stresses. Or when luck runs out and interrupts bunch up.
Unlike most other engineered systems our real-time devices don't have fuses that blow when something goes wrong. Instead of a controlled shutdown or fallback to a less-capable mode, firmware completely collapses in an unpredictable way.
There are some great tools available to help give insight into the timing behavior of our systems. SEGGER's trace probes are great adjunct's to one's engineering bench. IAR has similar products (though information can be hard to find; getting good tech details from their web site can be frustrating). I figure tools are cheap compared to developer's salaries and the risks of releasing a product that isn't reliable. But I wonder how many of us take advantage of the tools that exist.
We're trained to think in the procedural domain (if-then, do-while). The time domain is no less important. |
| Failure of the Week |
Donald Bailey sent this message from the future:
Leon Baždar's Windows 10 laptop is telling him (in Croatian) that 255% of his battery is available (We need this magical tech in EVs!):
Have you submitted a Failure of the Week? I'm getting a ton of these and yours was added to the queue. |
| Jobs! |
Let me know if you’re hiring embedded
engineers. No recruiters please, and I reserve the right to edit ads to fit the format and intent of this newsletter.
Please keep it to 100 words. There is no charge for a job ad.
|
| Joke For The Week |
These jokes are archived here.
Al-Gebra is a problem for us," the Attorney General said.
"Al-Gebra has terrorized many young people for years. They
derive solutions by means and extremes and sometimes go off on
tangents in search of "absolute values". "They use secret code
names like 'X' and 'Y' and refer to themselves as 'unknowns,'
but we've determined that they belong to a common denominator of
the axis of medieval with coordinates in every country."
Searches for other incriminating possessions revealed traces of
calculus & trig, the threats of which are being examined by
special agents. Solutions are not expected soon.
As the Greek philosopher Isosceles used to say, "There are 3
sides to every triangle.'" When asked to comment on the arrest,
the President said, "If God had wanted us to have better
weapons of maths instruction, he would have given us more
fingers and toes." |
| About The Embedded Muse |
The Embedded Muse is Jack Ganssle's newsletter. Send complaints, comments, and
contributions to me at jack@ganssle.com.
The Embedded Muse is supported by The Ganssle Group, whose mission is to help embedded folks get
better products to market faster. |
