Temporary Highs and Lows – Regression to the Mean

You are home on a Sunday afternoon, watching your favorite NFL team tear up the opposition.  Their offense is clicking on all cylinders and has registered 17 points as the first quarter comes to a close.  What expectation should you have for the remainder of the game?  Are they likely to repeat their performance and score 17 points in quarter 2?  Can they be expected to continue at this torrid pace for the remainder of the game?

Experienced football fans will intuitively know that the answer to both questions is a resounding no.  Looking back at statistics (courtesy of Pro-Football-Reference.com), teams have scored 17 or more points in the first quarter of a game 283 times (timeframe 1970 to date).  Of those 283 occurrences, teams have scored 17 or more points in the second quarter only 15 times.  Additionally, no team has ever kept that pace for an entire game.

The phenomenon described above is a classic example of a statistical law known as regression to the mean (RTM).  It is a law that is routinely disregarded or misunderstood, leading to poor conclusions and suboptimal decisions in our personal and professional lives.

RTM is the tendency for unusual or extreme performance to be typically followed by normal or average performance.  In the football example, scoring 17 points is an unusually strong performance.  As the historical data showed, only a tiny percentage of teams maintain that level of performance for an additional quarter.  All others regress to a more average scoring output.  This pattern of occasional extreme performance followed by more typical performance is normal for most processes.  It is found everywhere from human performance to the weather to corporate results.

RTM was first identified as a statistical law by the Victorian era polymath, Sir Francis Galton.  Galton was a half-cousin of Charles Darwin and was also responsible for coining the term nature vs. nurture.  He rightly observed that children of very tall parents would typically be taller than average, yet shorter than their parents.  Similarly, the children of very short parents would be shorter than average, but taller than their parents.  He initially termed the phenomenon reversion to mediocrity.

Nobel prize winning researcher Danny Kahneman stumbled on an example of RTM when he was teaching a class on the psychology of training to flight instructors in the Israeli air force.  He was explaining how rewards rather than punishments were a more effective system of motivation for trainees.  An instructor objected, saying that students always did better after being chided for a poor performance.  Conversely, when praised after a very good performance, they uniformly did worse.  Kahneman realized that the instructor was misinterpreting RTM.  RTM would tell us that a superior performance was likely to be followed by a less impressive performance and a poor performance by a more satisfactory one.  This should happen irrespective of the tactics used by the instructor.

Kahneman’s observation of coaching styles and student performance was duplicated in a clever scientific study by P.E. Schaffner in 1985.  Subjects were seated at a computer where their goal was to encourage a hypothetical student to arrive to school on-time at 8:30.  The “student’s” arrival time was pre-programmed to be between 8:20 and 8:40.  After each “arrival”, the subject was given the option to praise, reprimand or do nothing.  However, the subject’s choice had no real effect on the student’s subsequent performance, which had been predetermined.  As human nature would predict, the subjects praised the student when arriving early and reprimanded them when arriving late.  Due to RTM, the students performance typically improved after being punished and worsened after being praised.  Resultantly, 70% of the subjects viewed reprimands superior to praise as an effective coaching tool.

People have an unfortunate tendency to see patterns in randomness.  They like for there to be a causal story explaining what is often simply natural variation.  Unfortunately, this results in the establishment of false connections between actions and results.  The department that led the company in performance last month is probably one of the better units in the firm.  However, they are likely, through normal variation, to have a decrease in performance the subsequent month.  This decrease in performance is often attributed to some personal or team failing.  For example, “Joe seems distracted lately” or “Department X seems to have lost their initiative.”

The normal variation described by RTM takes two different forms.  First, any person, group or system does not sustain extreme performance. Let’s consider a basketball player who normally makes 75% of his free throws.  In his previous game, he shot an amazing 10 for 10, or 100%.  RTM would predict that in his next game he would be likely to return to his average of 75%.

Random external factors can also be a factor in extreme results.  The team that scored 17 points in the first quarter may be the beneficiary of random favorable events that are unlikely to reoccur.  For example, a fumble may have bounced directly into the hands of a defender with a clear path for a touchdown.  Alternatively, an official may have made an incorrect call, resulting in an interception and touchdown that your team didn’t deserve.

Having an understanding of RTM can make you a better analyst and forecaster.  When analyzing results or making predictions, make sure that you take RTM into account.

  • Be skeptical of unusual or extreme performance.  It is frequently an aberration that will revert to more normal performance.
  • Make sure you understand the historical average and trend for the metric in question.
  • Recognize that all processes are subject to normal variation.  Changes in performance are not necessarily indicative of a change in underlying capability.

 

This entry was posted in Cognitive Bias, Probability/Statistics, Psychology. Bookmark the permalink.

9 Responses to Temporary Highs and Lows – Regression to the Mean

  1. Sam McNerney says:

    Nice article Dan, I enjoy reading about this stuff.

    Here’s what I’m thinking. I’ve been reading a lot about happiness studies lately – positive psychology stuff. And I am discovering that many psychologists now believe that happiness is about 50 percent genetics. This means that no matter what you do in life (minus traumatic life-changing stuff) you will always return to your default state. As Jonathan Haidt explains:

    “In the long run, it doesn’t matter what happens to you. Good fortune or bad, you will always return to your happiness set point – your brain’s default level of happiness.”

    So perhaps we could think of happiness in terms of “regression to the mean.” Just as the Israeli pilots always returned to their default ability, everyone returns to their default state of happiness. The only different, of course, is that you can improve your skills as a pilot much more than you can improve your happiness (no innate forces preventing your pilot skills) Perhaps I am speaking to metaphorical but you get the idea.

    • Dan says:

      Sam,
      Thanks for the insightful comment. You bring up an interesting point of comparison. I agree with with the concept of a split role between nature and nurture, varying by individual trait. I would argue that even the pilots are constrained to some degree by genetics. Vision, balance, reflexes, and fearlessness are all traits with strong genetic components. Certainly, training creates huge improvements in performance. But my guess is that a decent percentage of folks could never become top fighter pilots due to some inherent limitation.

  2. Pingback: Human Nature – Friend and Enemy of Process Improvement | Invisible Laws

  3. Pingback: Beats By Dre Pro sale

  4. Pingback: Beats Solo HD Headphones

  5. Pingback: Cheap Beats

  6. Pingback: SQBynNcQ

  7. Pingback: IijQVjkQ

  8. Pingback: NqsNWpfL

Comments are closed.