Regression Towards the Mean - Mental Model
Value a good track record over one-time success stories.
Last weekend I ran my last half-marathon of the year, and a magical thing happened.
I slowed down for a few seconds to lower my heart rate. Then a guy overtaking me tapped my shoulder and said "There are just 3k left, don't slow down”. The magic is, it felt like he gave me extra energy that I didn’t have. I started to run faster.
Was this magic? I wanted to believe it was because of the stranger's motivating words. But it probably was a regression to the mean. My heart rate was unusually high at that moment and the brief rest brought it back to a level at which I was able to recover.
External praise can sometimes influence physical performance. However, in my case, a brief rest might have been equally as helpful as the encouragement from this runner.
Regression towards the mean belongs to the field of statistics. It's a pattern where very high or low measurements in one test tend to be closer to the average in the next test. It happens because extreme values are often due to unusual conditions, so when we measure again without changing anything, the results are likely to be closer to the average.
Examples
We can observe regression to the mean whenever we monitor extreme values that might be too good to be true or are far from normal operating levels. The key is to repeat the measurements without changing conditions and observe the results.
Sports
For sports performance, years of training result in an established level of skill. For example, points scored during basketball games vary from player to player, but over time, it has been observed that, despite having good and bad games, players tend to return to their average scores.
Extremely good or bad performance in one game will likely be followed by a performance closer to the player's average ability in the next.
Performance Improvement Plans
Players in sports have an established set of skills which do not change rapidly. Similarly, engineers have a set of skills and average performance.
Performance improvement plans are introduced as good practice to give employees a chance to improve before a decision is made to part ways. These plans help to closely monitor performance with objective goals for a specific period. However, regression towards the mean can play a significant role in this process:
An engineer had been performing below their average for a while, and the performance improvement plan appeared successful as the engineer returned to their usual performance (the underperformance was caused by external factors)
An engineer was performing poorly, and the performance improvement plan didn't help — they were consistently below the requirements for the position.
These plans can be viewed as a way to allow time for performance to progress towards the mean.
I'm not convinced that significant improvements in someone's overall skill set can be achieved within relatively short time spans. However, sometimes it's worth trying.
Engineering
Observations on regression towards the mean:
Extremely good performance of the team might not mean that the next sprint will be delivered equally well.
If QA found a record-high number of bugs in one day, it’s unlikely to happen the next day.
If we had zero incidents today while the average is three, we should prepare for incidents tomorrow and not assume they won't happen.
Imagine your team performed unusually badly in the last sprint:
You might feel you can help by preparing a speech to motivate them, so you did it. The next sprint is much better.
It was thanks to the speech, right? No, not necessarily. They may have simply returned to their average good performance. We often mistakenly attribute changes caused by regression towards the mean to our decisions or actions. This leads to false beliefs about causation.
Summary
Regression towards the mean helps us understand unusually good or bad events and behaviours. It explains how luck and performance balance over time.
Be sure to:
Do not overreact. When extremes happen we tend to draw conclusions. But regression towards the mean reminds us to be cautious.
Value a good track record over one-time success stories.
Set realistic expectations. Best-ever sprints or performance may not last for the next iteration.
Thanks for reading,
— Michał
P.S. My interest in mental models has been growing over the years thanks to Farnam Street. Their podcast, The Knowledge Project, and their books. FS covered Regression Towards the Mean using great examples from Daniel Kahneman’s book.
Post Notes
Corresponding topic: Correlation Does Not Imply Causation — Don't be fooled by good-looking charts.
Catalogue of Mental Models — Power Up Your Brain
Discover Weekly — Shoutouts
Articles that might help you explore new perspectives, which I’ve read recently:
"The importance of data when making decisions in the engineering industry" by
and"Shut your mouth and make room for others" by
This was interesting Michał, thanks for sharing your story.
I like this topic because I learned a bit about statistics and probability.
It also made me recall something I'm dealing with right now while helping a friend.
In general, I try to stay away from wondering about what could have happened if this or that happened in the past. Learning from some recent medical complications and trying to recover from them, fantasizing about the different possible outcomes didn't help.
Let's say someone had an operation and now things are settled and they are recovered, but the operation also had a long-term effect, let's say it is a scar. They are bothered by the scar and think about alternative realities where a million things that now function in their body correctly, still do, they are well but the scar isn't there.
I see this as an example of the conjunction fallacy. There are simply too many things that had to be in a specific way to end up healthy and recovered. And thinking that "oh yeah, but if I could just flip that switch in the past" and get the same results, I think that's misleading because of the immense number of variables.
Thanks for the mention!
Great article and as always great job connecting it back to working in tech. I have 2 thoughts about Regression to the Mean:
1. It's pretty hard to know or measure your mean.
2. Your mean could trend up.