What is Outcome Bias?

A video explaining Outcome Bias by Rolf Dobelli the author of The Art of Thinking Clearly on the IDFC Mutual Fund Youtube Channel.

WHAT IS OUTCOME BIAS AND WHY YOU SHOULD NEVER JUDGE A DECISION BY ITS OUTCOME

A quick hypothesis: say one million monkeys speculate on the stock market. They buy and sell stocks like crazy and, of course, completely at random. What happens? After one week, about half of the monkeys will have made a profit and the other half a loss. The ones that made a profit can stay; the ones that made a loss you send home. In the second week, one half of the monkeys will still be riding high, while the other half will have made a loss and are sent home. And so on. After ten weeks, about 1,000 monkeys will be left – those who have always invested their money well. After twenty weeks, just one monkey will remain – this one always, without fail, chose the right stocks and is now a billionaire. Let’s call him the success monkey.

How does the media react? They will pounce on this animal to understand its ‘success principles’. And they will find some: perhaps the monkey eats more bananas than the others. Perhaps he sits in another corner of the cage. Or, maybe he swings headlong through the branches, or he takes long, reflective pauses while grooming. He must have some recipe for success, right? How else could he perform so brilliantly? Spot-on for twenty weeks – and that from a simple monkey? Impossible!

The monkey story illustrates the outcome bias: we tend to evaluate decisions based on the result rather than on the decision process. This fallacy is also known as the historian error. A classic example is the Japanese attack on Pearl Harbor. Should the military base have been evacuated or not? From today’s perspective: obviously, for there was plenty of evidence that an attack was imminent. However, only in retrospect do the signals appear so clear. At the time, in 1941, there was a plethora of contradictory signals. Some pointed to an attack; others did not. To assess the quality of the decision, we must use the information available at the time, filtering out everything we know about it post-attack (particularly that it did indeed take place).

Another experiment: you must evaluate the performance of three heart surgeons. To do this, you ask each to carry out a difficult operation five times. Over the years, the probability of dying from these procedures has stabilised at 20%. With surgeon A, no one dies. With surgeon B, one patient dies. With surgeon C, two die. How do you rate the performance of A, B and C? If you think like most people, you rate A the best, B the second best, and C the worst. And thus you’ve just fallen for the outcome bias. You can guess why: the samples are too small, rendering the results meaningless. You can only really judge a surgeon if you know something about the field, and then carefully monitor the preparation and execution of the operation. In other words, you assess the process and not the result. Alternatively, you could employ a larger sample, if you have enough patients who need this particular operation: 100 or 1,000 operations. For now it is enough to know that, with an average surgeon, there is a 33% chance that no one will die, a 41% chance that one person will die and a 20% chance that two people will die. That’s a simple probability calculation. What stands out: there is no huge difference between zero dead and two dead. To assess the three surgeons purely on the basis of the outcomes would be not only negligent but also unethical.

In conclusion: never judge a decision purely by its result, especially when randomness or ‘external factors’ play a role. A bad result does not automatically indicate a bad decision and vice versa. So rather than tearing your hair out about a wrong decision, or applauding yourself for one that may have only coincidentally led to success, remember why you chose what you did. Were your reasons rational and understandable? Then you would do well to stick with that method, even if you didn’t strike lucky last time.

Next:
Paradox of Choice – LESS IS MORE

Similar Biases:
Sunk Cost Fallacy – WHY YOU SHOULD FORGET THE PAST
Swimmer’s Body Illusion – DOES HARVARD MAKE YOU SMARTER?
Hindsight Bias – WHY YOU SHOULD KEEP A DIARY
Illusion of Skill – THE BOAT MATTERS MORE THAN THE ROWING

Previous:
Regression to Mean – THE DUBIOUS EFFICACY OF DOCTORS, CONSULTANTS AND PSYCHOTHERAPISTS

The above article is from the book The Art of Thinking Clearly by Rolf Dobelli. The article is only for educational and informative purposes to explain and understand cognitive biases. It is a great book, definitely worth a read!