All models are wrong, but some are useful — George Box quote

One of my favourite statisticians’ quotes is British statistician George Box’s point that:

“All models are wrong, but some are useful.”

— George Box

His point, as I understand it, is simply that any model is inherently a simplification and approximation of reality. It will never capture reality in its entirety.

This applies to your formula of expected revenue and expenses that you drag down cells in a spreadsheet. And it applies to our largest, immensely complex, weather and climate models with millions of individual variables and data points. They can never be complete. And while we should keep this in mind for the limitations of all our models, forecasts, predictions and explanations, it doesn’t stop them from, many times, being useful.

While you can, and many people do, argue with the premise of models being “wrong,” I like the quote because it reminds me to be humble and duly sceptical of any models I read about or put together. While their accuracy varies enormously, they are all fallible.

It also reminds me that any model has a purpose—emphasising some features of reality and ignoring others.

On Maps

While a map is not a model in the statistical sense that George Box likely had in mind when he uttered the phrase, I find that maps are a great example of selective focus.

In the sketch, you can see the messy complexity of the world, the traditional map where buildings and roads become marks on paper, and the simplification of a bus route’s sequential stops.

Other times, maps may focus on:

• Human geography: streets, buildings, shops
• Tourist attractions
• Relief, elevation and geographic features
• Water courses and watersheds
• Traffic and travel times
• Political boundaries

And any number of other things. They are all models representing some aspects more accurately at the expense of others. As the saying goes, “The map is not the territory.”

Other Examples

Some other examples of simplified, yet still useful models.

• Weather: imagine trying to condense all the complexities of a day’s weather in the UK into a single icon. Not so easy. And yet, it can still help you decide if you should bring an umbrella or postpone a sailing trip.
• The Mercator projection: Useful for sailors. Massively distorted at high latitudes.
• Naismith’s Rule: for estimating walking time in the mountains based on distance and elevation
• An orthographic drawing of a house or product: may highlight relevant features for construction—wiring, plumbing, structural integrity—while ignoring textures, colours, imperfections or cost.
• The Bohr model of the atom: electrons orbiting like planets around the nucleus. Imperfect, but useful.
• Normal distributions: such as you might see in people’s height, are rarely, if ever, perfectly normal and yet can still be useful approximations.
• Supply and demand curves: approximate buyer and supplier behaviour while making assumptions about rationality and the availability of information.
• The London Underground map: famously simplified to perpendicular and angled lines, transforms travelling on the tube and confuses walking distances.

A model is a tool, and its usefulness depends on what you are trying to do. If you’re trying to plot courses on a flat map, then the Mercator projection is extremely helpful. If you’re trying to understand how large Greenland is compared to other countries, then it’s misleading.

Box advocates using economical models that allow us to interrogate practice while staying aware of where they may be importantly wrong.

It’s not lost on me that I present a lot of simpler models than reality in Sketchplanations. I think that’s because, when we consciously acknowledge their imperfections and limitations, a good model, like a good framework, can be just so helpful. As Larry Keeley said, “Building a good framework is like cutting cubes out of fog.”

Related Ideas to All Models are Wrong

Also see:

• Without data, you’re just another person with an opinion — W. Edwards Deming
• The Metrics Onion
• VUCA: Volatility, Uncertainty, Complexity, Ambiguity
• Sneaky averages
• Measures of central tendency: Mean, median, mode
• Correlation is not causation
• In theory, practice is the same as theory, but not in practice
• Sampling bias
• Chihuahua syndrome
• Science and engineering: what’s the difference?
• The mathematics of everyday life

Rick Wicklin wrote a helpful write-up of his investigations on the quote: Did George Box say, "All models are wrong, but some are useful"? (Spoiler: Yes)