In a previous post, I covered a phenomenon in statistics known as Simpson’s Paradox. This “paradox” causes people to misinterpret data and draw incorrect conclusions. While Simpson’s Paradox is a common error, it is far from the only mistake that people make when drawing statistical conclusions. One general area that people seem to struggle with is understanding the nature of randomness. In this post, I’ll describe some common fallacies regarding randomness and how an understanding of those fallacies can lead you to better personal and professional decisions.
The first and most basic error people make regarding randomness is known as the Gambler’s Fallacy (or Law of Averages). This fallacy is the erroneous belief that previous outcomes of a random event effect the odds of a subsequent event. For example, the odds of a coin flip coming out heads (for a “fair” coin) are 50%. If the coin is flipped 5 times and results in 5 straight heads, the odds of a subsequent toss resulting in heads is still 50%. The Gambler’s Fallacy leads to an assumption that tails would be highly likely on the 6th flip because its was “overdue”. When looking at deterministic probabilities like a coin toss, the odds of each flip are separate and distinct from any other tosses.
Another mistake that is very common is something known as the Hot Hand Fallacy. It is seen when people observe an individual’s performance in a given domain. It is effectively the opposite of the Gambler’s Fallacy. Let’s say a basketball player has made his last 5 shots. There is a natural tendency to see his performance as “hot”. The average observer would think that the chance the player will make his next shot is far in excess of his average shooting percentage. Surprisingly, a detailed study of the NBA by Cornell professor Tom Gilovich found no evidence of this “hotness” dynamic. The study concluded that the chance of a player making a shot was independent of whether he had made or missed the previous shot.
Both Hot Hand Fallacy and Gambler’s Fallacy are examples of the tendency that people have to misunderstand the nature of randomness. Humans are pattern seeking beings. That is, we have a tendency to look for connections and causality when we see a series of phenomena. This tendency served us well for thousands of generations in prehistoric times. For example, learning to associate a rustling in the grass with the presence of a snake is a helpful trait. Even if it turns out that 90% of the time it’s simply the wind blowing. But those same developed instincts that allowed for our survival work against us in the modern world.
People have an expectation that randomness will be seen with virtually no repetition of results. A pattern such as HTHTTHTHHTHT (representing heads or tails for a coin toss) would fit most people’s expectation of randomness. In reality, it would be common to see greater streaks in the results. For example, when tossing a fair coin 100 times there is a 97% chance that there will be a streak of 5 heads or tails in a row. In fact, there is close to a 50% chance that you would see a streak of 7 heads or tails. Most people would find these probabilities very counter intuitive.
The need for people to see meaning from patterns can lead to inappropriate conclusions. Rather than seeing the possibility of random coincidences, we are more likely to look for connections. Let’s look at how these issues apply to the real world of judgement and personal decision making:
Imagine we have a department that has 100 measurable activities per month. It could be an account management team measured on customer satisfaction for 100 clients. Alternatively, it could be a software development team with 100 open milestones. Maybe its a help desk, rated on call handling for 100 different shifts. Let’s assume that historically the failure rate for any of these activities is a low 1%. Over the course of a year management would expect approximately 1 “failure” per month from each team. What’s the likelihood that a particular team would have 3 or more failures in a particular month? For one team, it would be approximately 8%. Now let’s say we’re an executive, responsible for 10 teams. The likelihood that 1 of these teams would have 3 or more failures in a given month is 56%. Furthermore, over the course of the year,there would be a 90% chance of at least 5 occurrences where a team exceeded the 3 failure mark in a given month.
The unlucky team that had the 3 failures would certainly draw scrutiny from management. Intuitively, it would seem unlikely that it was a random event. The human instinct for pattern seeking would look to quickly solve the “mystery”. The immediate focus would be on the competence of the team or its immediate supervisor. While it is prudent to investigate patterns of sub-par performance, it’s important to understand that some of these scenarios will arise simply from random variation. The manager’s role is to understand the likelihood of the failure, identify extenuating circumstances, and analyze the team’s historical performance. All these factors need to be weighed collectively to determine if the team is having performance issues or is the victim of normal variation.
Having a healthy respect for randomness and an understanding of basic probability theory are helpful skills for any professional. It can enable us to avoid being misled by our inherent need to see patterns. By applying these concepts, we can draw more accurate conclusions from data, and avoid confusing randomness with either incompetence or superior capabilities.