Safety is Hard

Summary: Building a safe system requires a level of systems thinking that may be impossible.

We bandy the phrase "safety-critical system" around, too often not reflecting on its meaning.

What does "safe" mean? Can you prove your system is safe? I doubt it, since that's rather analogous to proving the absence of bugs. There's really an epistemological problem with the notion of safety, since one can only create arguments for risks one understands, not the entire universe of possible risks.

A system - a black box, instrument, device or other stand-alone device might be "safe," but could be a disaster in practice. That system is undoubtedly just one component in a bigger product, and its interaction with the rest of the world may not be safe. The rest of the world includes people, and people are notoriously competent at injecting an idiot factor that defies most safety reasoning.

A couple of weeks ago I was on a long-haul flight and was pleased that the seat had a 110 VAC outlet to power the laptop. A 14 hour hop is about three times longer than my laptop battery lasts.

But early in the flight I was engrossed in Jean Smith's new Eisenhower biography and sort of oblivious to my surroundings. Eventually looking up I noticed that my seatmate, a rather elderly Chinese woman, had her earbuds on and was trying to insert the 1/8" connector. into the power outlet!

A safety case for the power outlet would probably figure on low-amp fuses, proper grounding, and other parameters. But who would factor in "elderly" and "earbud"? Even more confounding, the outlet was of the North American three-prong configuration, which was possibly foreign to this Chinese national. (I once knew a Thai woman who had grown up in a bamboo shack with no electricity - it's probably dangerous to assume any familiarity with technology when catering to the general public).

Was she an idiot? Of course not. On reflection, it's sort of logical to expect the audio socket to be near the screen instead of hidden on the armrest.

The 110 VAC outlet is next to the screen

Published May 16, 2012