Natural variation and decision making

Imagine you are in your 9th grade. You scored 80%, everyone expected you to. The teacher praises you. Your friend scored 40% and the teacher scolds him. Next exam. You score 76%. Your friend gets 45%. What just happened?

It is seemingly logical that praising you got you to slack off and lose your grade while your friend benefited after scolding. The thing is, while it appears that the teacher influenced the grades, such a fluctuation is likely to be a natural fluctuation. Plain and simple changes that depend on so many variables that they appear random to us. 

There is a very high chance that the teacher’s scolding and praising changed nothing. This is called the regression fallacy^[1]. It is based on a couple of things. One, there is a tendency for the score (or grade, or any variable) to move toward the average after an extreme score. Two, erroneous attribution of events to a controllable cause. These innocuous random and natural fluctuations are often underestimated. 

Natural variations and fluctuations are significant causes that we often underestimate

Where would you see natural fluctuations being underestimated?

1. Temperature
2. Social media responses – likes, shares, traffic, comments
3. Exam marks
4. Road traffic
5. Biology- height of progeny, weight, hair fall
6. Cryptocurrency (my newest favorite)
7. Plant Yield
8. Pain and medical symptoms
9. Pain and medical symptoms

Humans like to find and attribute causes for minor changes. Take, for example, road traffic! With a mild increase in traffic, how often do humans try to figure out a cause?

At least here in India, people often conclude and confirm arbitrary reasons like the weekend is here, cricket match, politician visits, etc. These happen all the time. Trees bear fruits, some years have a slight increase, sometimes a slight decrease. Human’s like to find reasons for events. *

It happens for a reason, there must be a specific cause for this….. Natural variation & fluctuations my friend. They happen. To us they are random. They happen at the genetic level, at the societal level, at the environmental level.

The statistical concept is known as regression to the mean^[2]. After we measure something to be extreme, the next measurement is likely to be closer to the average expected. The cognitive concept is called the regression fallacy, an informal thinking error.

Natural variations are underestimated to such an extent that we waste money to find an answer that doesn’t exist. They often explain a lot of things such as changes in monthly sales or marketing reach. The boss sees a difference or a small drop in numbers, he/she frantically hires an agency or borrows employee time, and spends money to find out the reason. Natural variation also is the simplest answer at times. That is Occam’s razor – The simplest answer tends to be the right one.

However, this post isn’t about that. If natural variation can explain a phenomenon, there is a high chance that that explanation is correct.  And I emphatically highlight – Do not underestimate natural fluctuations and variations.

Now that we know what natural variation entails, let us look at how this leads to poor decisions.  Back to imagining.  You are having a conversation with your significant other. He/she has natural variation accounting for being a little late. Is this understandable if it happens once every few days? Let us also assume that there is no malice and that your S/O isn’t tardy (like occam’s razor, this is another philosophical razor called Hanlon’s razor).

Would you take a decision on your impression of the SO? Would you raise a problem based on this natural variation? Will you decide to escalate the matter into an argument?  You are running late for work. Natural variation accounts for leaving your house 10 mins late. Road traffic adds another 10 mins. No one scolds you, but you reflect on this and choose to change your work route, or you spend time in rushing through your work- leading to slightly poorer outputs.

These are some micro-decisions you take on the fly. After deciding on rushing through the work to finish before lunch or changing travel routes or skipping shampoo, you’d expect to have enough time to reach work. Remember, natural variation caused the delay, and you decided to rectify some aspects of your routine. But those are not exempt from natural variation. Those decisions would lead to events that are also subject to natural variation.

Suppose you score poorly on one exam. Just slightly poor. Say your personal standard is 75%. You earn 72%, would you choose to join a tutor because you scored poorly? That would lead to a lot of time being wasted. Moreover, if you’ve been consistent around 75%, 72% can be accounted for by natural variation.

Let us say you raise 2 children with your own genetic material. One turns out to be taller than the other. You see this difference prominently during their teens. Natural variation, at the genetic level, may have caused the height difference. However, some parents might think there was a dietary problem, perhaps one kid was eating poorly at school. Would you spend time and money on hiring a dietician and changing the kid’s routine?

An overarching theme in these examples is – Natural variation leads to events and phenomena which we try to modulate or control by taking decisions that are unnecessary. Those decisions are often a waste of time and resources.

Sources