The Scarcity of Certainty - Uncertainties - Better Business Decisions from Data: Statistical Analysis for Professional Success (2014)

Better Business Decisions from Data: Statistical Analysis for Professional Success (2014)

Part I. Uncertainties

In this world nothing can be said to be certain, except death and taxes.

—Benjamin Franklin

We need to understand the reasons why statistics embodies uncertainties. This will give us a feel for what statistics can do and what it cannot do, what we can expect from it and what we should not expect. This will prepare us for critically viewing the statistics and the conclusions from them that we are presented with. Some understanding of basic probability, which is required to appreciate uncertainty, is presented without assuming any previous knowledge on the part of the reader.

Chapter 1. The Scarcity of Certainty

What Time Will the Next Earthquake Be?

On the twenty-second of October, 2012, in Italy, six geophysicists and a government civil protection officer were sentenced to six years in prison on charges of manslaughter for underestimating the risk of a serious earthquake in the vicinity of the city of L’Aquila. Following several seismic shocks, the seven had met in committee on March 31, 2009, to consider the risk of a major earthquake. They recorded three main conclusions: that earthquakes are not predictable, that the L’Aquila region has the highest seismic risk in Italy, and that a large earthquake in the short term was unlikely. On April 6, a major earthquake struck with the loss of more than 300 lives.

The court’s treatment of the seismologists created concern not only among seismologists working in other countries, but also among experts in other fields who are concerned with risk assessment. All seven filed appeals in March 2013, but it seemed unlikely that there would be a ruling on the case for some years. Whatever that may be, the case highlights the difficulties and the dangers in making decisions that have to be based on data that are statistical. If it is decided that an event is unlikely, but it then occurs, was the decision wrong? The correct answer is no, because unlikely events do happen—but there is a common misperception that the answer is yes.

An unfortunate consequence of this perception is either that it becomes more and more difficult to find anyone who is prepared to make a decision where risk is involved, or else that decisions become based on worst-case scenarios and thereby frequently create unwarranted disruption and expense. There are instances of this in relation to health and safety legislation. Some school teachers have refused to take children on school field trips in case an accident occurs. Warnings are posted at gas stations instructing customers not to use their mobile phones near the pumps, although there has never been a reported case of a fire having been caused by their use. Homeowners are hesitant whether to clear snow from their sidewalk, for fear inadequate clearance might result in a passerby suffering a fall that could lead to a claim for compensation. A traditional British game,conkers, played by schoolchildren, involves two contestants, each of whom has a horse chestnut at the end of a length of string. Taking turns, each player attempts to shatter the other’s suspended conker by whirling his or her own conker against it. The ­playground is scattered with fragments! The game has been played for generations with no public concern, but some schools in the UK now insist that the children must wear gloves and goggles.

Even the trivial decisions we make every day are often based on statistical data and have a degree of uncertainty. We have a meeting to get to, and we decide to catch the 9:20 AM train. We know from past experience that the train is usually on time, and we have never known it not to run. If, on the other hand, we knew that the train were late more often than not, then we would probably decide to catch an earlier one. Of course, we cannot be certain that the train will be on time. We cannot even be certain that the meeting will take place.

There are, of course, decisions we can make that are based on matters we can be certain of, but these are fairly inconsequential routine activities or observations encountered on a regular basis. The decisions are based on well-established facts. The difficult decisions—the important ones—usually involve issues that are uncertain to some degree.

Proof is often adduced as justification for decisions, but even here we have to be careful to recognize that qualifications or exceptions may apply. A court of law demands proof, but there is the qualification that it should be proof beyond a reasonable doubt. Scientists are said to be in the business of proving things by experiment and observation. In reality, scientists are able to do no more than provide theories that explain the way things work and allow predictions to be made. These theories are always considered to be provisional. Should they fail to make correct predictions, they will be modified or replaced. We frequently hear that science has proved this, that, and the other, when it has done no such thing. It has, of course, provided explanations and numerous correct predictions, from which our knowledge has increased and on which our technological advances have been based.

A statement may be true by definition. In mathematics, we define 2 to be the sum of 1 and 1, so that 1 plus 1 is always equal to 2. Other numbers are defined accordingly and, by defining mathematical processes—multiplication, taking the square root, and so forth—in a precise way, we can ensure that our mathematical proofs are indeed proofs. The rules of logic, based again on definition, lead to proofs. A valid syllogism—for example, “All cows eat grass; this animal is a cow; therefore, this animal eats grass”—is beyond dispute, though the truth of the conclusion does, of course, depend on the truth of the initial propositions. Statements that are true by definition do not add to our knowledge—for they are simply expressing it in a different way—but they may add usefully to our understanding.

Common sense is frequently used as a substitute for proof. We recognize a proposition as obvious, and we are then critical when we see investigations sponsored to show it to be correct. A waste of time and money, we conclude. But, as Duncan Watts (2011) demonstrates comprehensively, although common sense is useful in guiding us through our numerous daily activities, it is quite unreliable when we make judgments on more complex matters. Indeed, we can often recognize both a situation and its opposite as being obvious, depending on which we are told is correct. Children are healthier now than they were forty years ago. That seems obvious enough: health care is now better and there is better guidance on diet. But what about the opposite proposition, that children were healthier forty years ago than they are now? Common sense tells us this also is true: children then had more exercise, less time in front of the computer screen, and a simpler healthier diet.

When we use the justification of common sense to simply reinforce our own prejudices, the consequences may be trivial; but when those responsible for important decisions base them on common sense, the consequences can be serious. Government policy, company strategy, and marketing initiatives, for example, affect the well-being of many people and may do so for a long time.

Daniel Kahneman (2013) has produced an extensive study of how we suffer from errors and make bad judgments as a result of what he refers to as fast thinking. Our intuition is rarely reliable, and we are easily unconsciously influenced by circumstances. Our feeling for how much we should pay for something is influenced by the asking price. If you are asked whether Ghandi was more than 114 years old when he died, you will suggest a higher age than if the question asked whether he was more than 35 years old when he died.

There is a somewhat different use of the word obvious. The conclusion from a piece of reasoning or a calculation may be obvious to you, but the person sitting next to you may not find it so or even understand it. When tutoring students, I avoid the word. What is obvious to one person may not be obvious to another. For that reason, you will not encounter the word in the remainder of this book.