The Automation Paradox

The Automation Paradox is that the better our machines get, the more we struggle when they fail.

When heading out for a hike in the woods, it's tempting to skip the map and compass and rely on our phones and apps for navigation. Yet when we encounter no signal or lose power, we can find ourselves in a sticky situation. Or perhaps, like me, you've come to rely on popping your destination into the satnav or Google Maps whenever you get in the car and have nearly forgotten the ability to navigate without it.

These situations illustrate the paradox of automation, where the more sophisticated and automated our machines and technologies become, the more bewildered we find ourselves when they inevitably fail. Or: the smarter the machines get, the dumber we might get.

In his book Messy , Tim Harford suggests three strands to the paradox as our machines get more sophisticated:

Automation covers up our mistakes, hiding our incompetence, meaning we may not learn to correct ourselves—consider autocorrect cleaning up our typos as we go.
When we rely on automation, we get less practice for our skills, so even highly skilled individuals may find their expertise diminishing—perhaps you've found yourself using your phone calculator for a trivial calculation.
When the easy scenarios are taken care of, failures may occur in complex or unpredictable ways that we may find especially difficult to recover from—like a subtle but persistent failure in the steering of a passenger plane, recovering from a skid on an icy road, or when you're deeply lost in the wilderness.

More sophisticated technology can even make it useless or more dangerous when it fails. Older cars used to be reparable forever. Now, if your vehicle fails, it's likely to need plugging in at the dealership to figure out what's up. How many electronic devices are thrown away because somewhere inside, some tiny loose connection or component makes the whole thing worthless? Were pilots and flight crews better prepared and able to improvise before the autopilot became ubiquitous?

I often wish our devices would fail more like an escalator or an electric toothbrush. If an escalator fails, you can still walk up it. If your electric toothbrush dies, you can still use it to brush your teeth. These could be called Technology-Enhanced Products, perhaps. But when most of our devices die, they're often rendered worthless.

Automation and sophisticated machines help me so much. I did use Grammarly to help check this post. I use Google Maps nearly every time I put in my destination to home, and I often use a calculator to check my maths. But I do pay attention to grammar corrections, bring a paper map when I can, and keep trying to do the maths in my head.

But as Tim Harford explains it , we still face "the paradox of automation: the better the machines get, the more bewildered we are when the machines fail."

Also see:

The bus factor
Normalisation of deviance
Chaos monkey
Know your tech
Jevon's paradox
The law of unintended consequences
The Dunning-Kruger effect (send me proof of purchase of Big Ideas Little Pictures, and I'll send the sketch to you)

More paradoxes:

Tolkein-style landscape inspired by the excellent Lord of Maps .

Published 21 Jul 2024