The Automation Paradox is that the better our machines get, the more we struggle when they fail.

When heading out for a hike in the woods, it's tempting to skip the map and compass and rely on our phones and apps for navigation. Yet when we encounter no signal or lose power, we can find ourselves in a sticky situation. Or perhaps, like me, you've come to rely on popping your destination into the satnav or Google Maps whenever you get in the car and have nearly forgotten the ability to navigate without it.

These situations illustrate the paradox of automation, where the more sophisticated and automated our machines and technologies become, the more bewildered we find ourselves when they inevitably fail. Or: the smarter the machines get, the dumber we might get.

In his book Messy, Tim Harford suggests three strands to the paradox as our machines get more sophisticated:

Automation covers up our mistakes, hiding our incompetence, meaning we may not learn to correct ourselves—consider autocorrect cleaning up our typos as we go.

When we rely on automation, we get less practice for our skills, so even highly skilled individuals may find their expertise diminishing—perhaps you've found yourself using your phone calculator for a trivial calculation.

When the easy scenarios are taken care of, failures may occur in complex or unpredictable ways that we may find especially difficult to recover from—like a subtle but persistent failure in the steering of a passenger plane, recovering from a skid on an icy road, or when you're deeply lost in the wilderness.

More sophisticated technology can even make it useless or more dangerous when it fails. Older cars used to be reparable forever. Now, if your vehicle fails, it's likely to need plugging in at the dealership to figure out what's up. How many electronic devices are thrown away because somewhere inside, some tiny loose connection or component makes the whole thing worthless? Were pilots and flight crews better prepared and able to improvise before the autopilot became ubiquitous?

I often wish our devices would fail more like an escalator or an electric toothbrush. If an escalator fails, you can still walk up it. If your electric toothbrush dies, you can still use it to brush your teeth. These could be called Technology-Enhanced Products, perhaps. But when most of our devices die, they're often rendered worthless.

Automation and sophisticated machines help me so much. I did use Grammarly to help check this post. I use Google Maps nearly every time I put in my destination to home, and I often use a calculator to check my maths. But I do pay attention to grammar corrections, bring a paper map when I can, and keep trying to do the maths in my head.

But as Tim Harford explains it, we still face "the paradox of automation: the better the machines get, the more bewildered we are when the machines fail."

The Dunning-Kruger effect (send me proof of purchase of Big Ideas Little Pictures, and I'll send the sketch to you)

Tolkein-style landscape inspired by the excellent Lord of Maps.

The Automation Paradox

There's an all you can eat buffet with all the flavours of the world, you can have whatever you want. This is going to be amazing! You love lasagna, but look at those fajitas. You go for the fajitas, but then your friend arrives with sushi which looks even better. Why didn't you get that?? You always screw up...

Barry Schwartz studied how more choice not only doesn't always help us choose better, but can also make us feel worse about what we got even if it was great. Reduced satisfaction arises from:

The opportunity cost of what we could have had and what was good about other options

Regret and anticipated regret of the choice we made

Self-blame when we think we are responsible for not doing as well as we could have

For an entertaining and educational 20 mins watch Barry Schwartz' TED talk on the paradox of choice.

The paradox of choice

A favourite of our kids is to play opposites day. It usually starts with something simple like where we say “No, they can’t have chocolate for breakfast,” and they then announce that it’s opposites day so that means they can have chocolate for breakfast. Then I say, but if it’s opposites day then that means you just said it’s not opposites day. Then they say, if it’s not opposites day then what they said is true so it is opposites day. After a while we end up being late for school.

This is a version of the liar paradox, also known as the Epimenides paradox after a chap from Crete who is supposed to have announced that all Cretans are liars. There are a bunch of versions of the paradox including the simpler one: this sentence is a lie.

Some other paradoxes: Jevon’s paradox, the coastline paradox, the Abilene paradox.

The liar paradox

The more transparent your workplace the less transparent your employees. In an age of trackers, wearables, online monitoring, workplace chat and always-on devices it turns out that not only does a more transparent workplace actually drive people to more secretive behaviours, but that adding a little more privacy actually may improve productivity also.

Very simply, the transparency paradox is the idea that increasingly transparent, open, observable workplaces can create less transparent employees.

See Harvard prof Ethan Bernstein’s The Transparency Paradox study (pdf) or I learned about it from the Freakonomics podcast: Yes, the Open Office Is Terrible — But It Doesn’t Have to Be.

The transparency paradox

The paradox of how groups of people are able to take actions that no one in the group actually thinks is a good idea. Though individually, we each may think that an idea is a bad one, we may go along with the group decision thinking that it’s just us that disagree when, in fact, no one thinks it’s a good idea.

It was coined by Jerry Harvey, a management professor, who illustrated the phenomenon with a fictional story of a family who ended up on a rubbish day out to Abilene that no one wanted to do.

On a hot afternoon visiting in Coleman, Texas, the family is comfortably playing dominoes on a porch until the father-in-law suggests that they take a trip to Abilene (53 miles north) for dinner. The wife says, “Sounds like a great idea.” The husband, despite having reservations because the drive is long and hot, thinks that his preferences must be out-of-step with the group and says, “Sounds good to me. I just hope your mother wants to go.” The mother-in-law then says, “Of course I want to go. I haven’t been to Abilene in a long time.”

The drive is hot, dusty, and long. When they arrive at the cafeteria, the food is as bad as the drive. They arrive back home four hours later, exhausted.

One of them dishonestly says, “It was a great trip, wasn’t it?” The mother-in-law says that, actually, she would rather have stayed home but went along since the other three were so enthusiastic. The husband says, “I wasn’t delighted to be doing what we were doing. I only went to satisfy the rest of you.” The wife says, “I just went along to keep you happy. I would have had to be crazy to want to go out in the heat like that.” The father-in-law then says that he only suggested it because he thought the others might be bored.

The group sits back, perplexed that they decided to take a trip that none of them wanted. They each would have preferred to sit comfortably but did not admit to it when they still had time to enjoy the afternoon.

More paradoxes: The coastline paradox, Jevon’s paradox, the liar paradox, the transparency paradox, the paradox of choice

Also see: groupthink, the bandwagon effect

Harvey, J. B. (1974). “The Abilene paradox: the management of agreement”. Organizational Dynamics. 3: 63–80. Excerpt as quoted from Wikipedia :) 

This is not to diss Abilene. I’m sure it’s a neat place.

The Abilene paradox

This is the fascinating observation that it’s not straightforward to say how long a coastline is. If you were to measure the coastline of a country by using a ruler on a globe, you would come out with a vastly different number than if you were to pace around the edge. The closer you look, the more wiggles and squiggliness you come across and instead of converging on a more accurate length, the coastline just keeps getting longer. The smaller your ruler, the longer it gets.

This was originally spotted, incredibly, in the 1950s, by an Englishman, Lewis Richardson, when trying to check a theory he had that the likelihood of war between countries depended on the length of their shared borders. Remarkably, he found that the quoted lengths of borders varied significantly. While measuring on maps at different scales he saw that the smaller scale map he used, or the smaller the width of his calipers he was measuring with, the length systematically increased. When looking at coastlines, instead of borders, some countries had wigglier coasts and so the length increased at a faster rate with the scale — for instance, Norway’s coastline, with it’s crinkly fjords, increases faster than Britain’s, which in turn increases faster than South Africa’s, as he zoomed in. The rate of this increase later became known as its fractal dimension.

Long after Richardson’s research, Benoit Mandelbrot published a paper How Long Is the Coast of Britain? Statistical Self-Similarity and Fractional Dimension that discussed how the wiggliness of something like a coastline at one scale can be repeated at smaller and smaller scales. The work led to the later term fractals. Many other things exhibit fractal-like behaviour such as river networks, borders, brains, frequencies, lightning or even the stock market.

More mind-bending mapping with the Mercator Projection, and the difference between Great Britain, the UK and the British Isles.

There’s a super section on this in Scale, by Geoffrey West.

The coastline paradox

Jevon's paradox is that improvements in fuel efficiency tend to increase fuel use, not decrease it as you might expect.

In 1865, a chap named William Stanley Jevons noted that while there had been a number of improvements in coal efficiency in steam engines and the like, the effect was to increase consumption instead of reduce it. This observation became known as Jevon's paradox.

While it’s not a surefire outcome, there are plenty of parallels like more efficient batteries leading to more overall battery use, washing machines leading to doing more washes rather than less time spent washing, or the ability to recycle some plastics leading to more overall plastic consumption.

In her book More Work for Mother, Ruth Cowan addresses the ironies of household technology and how improvements in the efficiency of household devices often led to more work, not less.

This is sometimes known as the rebound effect.

Get my new weekly sketch in your inbox

Category: paradox