What is Overconfidence Effect? | Understanding Cognitive Biases

A video explaining Overconfidence Effect by Rolf Dobelli the author of The Art of Thinking Clearly on the IDFC Mutual Fund Youtube Channel.

WHAT IS OVERCONFIDENCE EFFECT AND WHY YOU SYSTEMATICALLY OVERESTIMATE YOUR KNOWLEDGE AND ABILITIES

My favourite musician, Johann Sebastian Bach, was anything but a one-hit wonder. He composed numerous works. How many there were I will reveal at the end of this chapter. But for now, here’s a small assignment: how many concertos do you think Bach composed? Choose a range, for example, between 100 and 500, aiming for an estimate that is 98% correct and only 2% off.

How much confidence should we have in our own knowledge? Psychologists Howard Raiffa and Marc Alpert, wondering the same thing, have interviewed hundreds of people in this way. They have asked participants to estimate the total egg production in the U.S., or the number of physicians and surgeons listed in the Yellow Pages of the phone directory for Boston, or the number of foreign automobiles imported into the U.S., or even the toll collections of the Panama Canal in millions of dollars. Subjects could choose any range they liked, with the aim of not being wrong more than 2% of the time. The results were amazing. In the final tally, instead of just 2%, they were off 40% of the time. The researchers dubbed this amazing phenomenon overconfidence.

Overconfidence also applies to forecasts, such as stock market performance over a year or your firm’s profits over three years. We systematically overestimate our knowledge and our ability to predict – on a massive scale. The overconfidence effect does not deal with whether single estimates are correct or not. Rather, it measures the difference between what people really know and what they think they know. What’s surprising is this: experts suffer even more from overconfidence than lay people do. If asked to forecast oil prices in five years’ time, an economics professor will be as wide of the mark as a zookeeper will. However, the professor will offer his forecast with certitude.

Overconfidence does not stop at economics: in surveys, 84% of Frenchmen estimate that they are above-average lovers. Without the overconfidence effect, that figure should be exactly 50% – after all, the statistical ‘median’ means 50% should rank higher and 50% should rank lower. In another survey, 93% of the U.S. students asked estimated themselves to be ‘above average’ drivers. And 68% of the faculty at the University of Nebraska rated themselves in the top 25% for teaching ability. Entrepreneurs and those wishing to marry also deem themselves to be different: they believe they can beat the odds. In fact, entrepreneurial activity would be a lot lower if overconfidence did not exist.

For example, every restaurateur hopes to establish the next Michelin-starred restaurant, even though statistics show that most close their doors after just three years. The return on investment in the restaurant business lies chronically below zero.

Hardly any major projects exist that are completed in less time and at a lower cost than forecasted. Some delays and cost overruns are even legendary, such as the Airbus A400M, the Sydney Opera House and Boston’s Big Dig. The list can be added to at will. Why is that? Here, two effects act in unison. First, you have classic overconfidence. Second, those with a direct interest in the project have an incentive to underestimate the costs: consultants, contractors and suppliers seek follow-up orders. Builders feel bolstered by the optimistic figures and, through their activities, politicians get more votes. We will examine this strategic misrepresentation later in the book.

What makes overconfidence so prevalent and its effect so confounding is that it is not driven by incentives; it is raw and innate. And it’s not counterbalanced by the opposite effect, ‘underconfidence’, which doesn’t exist. No surprise to some readers: overconfidence is more pronounced in men – women tend not to overestimate their knowledge and abilities as much. Even more troubling: optimists are not the only victims of overconfidence. Even self-proclaimed pessimists overrate themselves – just less extremely.

In conclusion: be aware that you tend to overestimate your knowledge. Be sceptical of predictions, especially if they come from so-called experts. And with all plans, favour the pessimistic scenario. This way you have a chance of judging the situation somewhat realistically.

Back to the question from the beginning: Johann Sebastian Bach composed 1127 works that survived to this day. He may have composed considerably more, but they are lost.

Next:
Chauffeur Knowledge – DON’T TAKE NEWS ANCHORS SERIOUSLY

Previous:
Hindsight Bias – WHY YOU SHOULD KEEP A DIARY

The above article is from the book The Art of Thinking Clearly by Rolf Dobelli. The article is only for educational and informative purposes to explain and understand cognitive biases. It is a great book, definitely worth a read!