You are home on a Sunday afternoon, watching your favorite NFL team tear up the opposition. Their offense is clicking on all cylinders and has registered 17 points as the first quarter comes to a close. What expectation should you have for the remainder of the game? Are they likely to repeat their performance and score 17 points in quarter 2? Can they be expected to continue at this torrid pace for the remainder of the game?
Experienced football fans will intuitively know that the answer to both questions is a resounding no. Looking back at statistics (courtesy of Pro-Football-Reference.com), teams have scored 17 or more points in the first quarter of a game 283 times (timeframe 1970 to date). Of those 283 occurrences, teams have scored 17 or more points in the second quarter only 15 times. Additionally, no team has ever kept that pace for an entire game.
The phenomenon described above is a classic example of a statistical law known as regression to the mean (RTM). It is a law that is routinely disregarded or misunderstood, leading to poor conclusions and suboptimal decisions in our personal and professional lives.
RTM is the tendency for unusual or extreme performance to be typically followed by normal or average performance. In the football example, scoring 17 points is an unusually strong performance. As the historical data showed, only a tiny percentage of teams maintain that level of performance for an additional quarter. All others regress to a more average scoring output. This pattern of occasional extreme performance followed by more typical performance is normal for most processes. It is found everywhere from human performance to the weather to corporate results.
RTM was first identified as a statistical law by the Victorian era polymath, Sir Francis Galton. Galton was a half-cousin of Charles Darwin and was also responsible for coining the term nature vs. nurture. He rightly observed that children of very tall parents would typically be taller than average, yet shorter than their parents. Similarly, the children of very short parents would be shorter than average, but taller than their parents. He initially termed the phenomenon reversion to mediocrity.
Nobel prize winning researcher Danny Kahneman stumbled on an example of RTM when he was teaching a class on the psychology of training to flight instructors in the Israeli air force. He was explaining how rewards rather than punishments were a more effective system of motivation for trainees. An instructor objected, saying that students always did better after being chided for a poor performance. Conversely, when praised after a very good performance, they uniformly did worse. Kahneman realized that the instructor was misinterpreting RTM. RTM would tell us that a superior performance was likely to be followed by a less impressive performance and a poor performance by a more satisfactory one. This should happen irrespective of the tactics used by the instructor.
Kahneman’s observation of coaching styles and student performance was duplicated in a clever scientific study by P.E. Schaffner in 1985. Subjects were seated at a computer where their goal was to encourage a hypothetical student to arrive to school on-time at 8:30. The “student’s” arrival time was pre-programmed to be between 8:20 and 8:40. After each “arrival”, the subject was given the option to praise, reprimand or do nothing. However, the subject’s choice had no real effect on the student’s subsequent performance, which had been predetermined. As human nature would predict, the subjects praised the student when arriving early and reprimanded them when arriving late. Due to RTM, the students performance typically improved after being punished and worsened after being praised. Resultantly, 70% of the subjects viewed reprimands superior to praise as an effective coaching tool.
People have an unfortunate tendency to see patterns in randomness. They like for there to be a causal story explaining what is often simply natural variation. Unfortunately, this results in the establishment of false connections between actions and results. The department that led the company in performance last month is probably one of the better units in the firm. However, they are likely, through normal variation, to have a decrease in performance the subsequent month. This decrease in performance is often attributed to some personal or team failing. For example, “Joe seems distracted lately” or “Department X seems to have lost their initiative.”
The normal variation described by RTM takes two different forms. First, any person, group or system does not sustain extreme performance. Let’s consider a basketball player who normally makes 75% of his free throws. In his previous game, he shot an amazing 10 for 10, or 100%. RTM would predict that in his next game he would be likely to return to his average of 75%.
Random external factors can also be a factor in extreme results. The team that scored 17 points in the first quarter may be the beneficiary of random favorable events that are unlikely to reoccur. For example, a fumble may have bounced directly into the hands of a defender with a clear path for a touchdown. Alternatively, an official may have made an incorrect call, resulting in an interception and touchdown that your team didn’t deserve.
Having an understanding of RTM can make you a better analyst and forecaster. When analyzing results or making predictions, make sure that you take RTM into account.
- Be skeptical of unusual or extreme performance. It is frequently an aberration that will revert to more normal performance.
- Make sure you understand the historical average and trend for the metric in question.
- Recognize that all processes are subject to normal variation. Changes in performance are not necessarily indicative of a change in underlying capability.