Do you think you can think critically? What does your intuition say?
Take this test to see how well you perform on one of the most popular cognitive tests. Think of them as simple brain teasers!
I’ll give some more context after you are done with the test.
Difficulty level: Medium
Explanation level: Detailed
The test: Analysis vs. Intuition
The Cognitive Reflection Test, made in 2005 by Professor Shane Frederick from MIT, USA, is a simple test with 3 questions that have an intuitive wrong answer and a critically thought-out correct answer. Answering correctly shows you can suppress wrong intuitive thoughts and analyze the problems.
Overriding an intuitive thought and thinking about it is called Cognitive Reflection[1]. It is essentially an intuition vs. analysis test. Getting all 3 right indicates the ability to think rationally and not make very biased decisions.
Each question has a “lure” that primes the brain to think in a certain way. That priming invokes automatic pathways for the brain to somehow wrongly apply logic to the details of the question. For example, in the lily pond question, the intuitive answer is applying the rule of “dividing by 2” to the only number in the question – 48. So people think the answer is 24. To not apply this rule, we have to override that priming.
Let’s take an easy example, which ChatGPT 3.5 has failed on.
Help me run this site with a donation :)
Question: I have 5 apples right now. I ate 2 yesterday. How many remain with me?
ChatGPT and most people use their intuitive thinking logic to assume this is a 5-2 question because we’ve been trained on such questions with an addition or subtraction mindset. This mindset is called a “mental set” – which is a pre-determined approach to solving problems. The numbers 5 and 2 become lures, but the logic of the question lies in when the apples were eaten.
The answer: 5 apples. It doesn’t matter how many you ate yesterday.
In another article, I show how logical thought requires context. If there is no context, we tend to make errors. The CRT is one unique test of logic that provides context. Because there is context, we know that the error in thinking is not because the questions are too abstract or vague, the errors arise because the brain focuses on “lures” in the questions.
You can test your knowledge of cognitive biases that lead to bad decisions here! It’s a small 9-item quiz.
The CRT moderately correlates with IQ and rational decision-making. It’s a popular way to test students’ critical thinking in interviews and competitive exams. It is also used to study the human brain under different conditions to see how intuitive reasoning increases or reduces and when critical thinking gets hard. Many different versions exist, but the original one is the most popular one.
There’s only 1 problem with the test now[2] – it’s so popular that many people with advanced degrees and job-hunting experience in urban settings have already taken it. Or seen a question or two on social media. If you know questions like these, please do comment and tell me, perhaps I can make another quiz (and credit you :))
Take the 9-question version of this test here and see what exactly goes into critical thinking.
How do people perform on this test?
A kid’s version of this test[3] (with 9 questions) shows that 5-year-olds typically get 1 to 3 questions right. And the score improves with age. 9-year-olds get 2 to 4 right, 12-year-olds get 3 to 5 right, and adults get 7 to 9 right. (I’ve listed these questions at the end, use them as your practice ground).
Adult men and women show a small difference[4] in performance when both haven’t been exposed to the test. Men, on average, score 1.57, and women score 1.42. The difference, researchers suspect, comes from a mathematical advantage men have through culture and upbringing and math anxiety that can worsen women’s performance.
Older people (around 60-70-year-olds) often perform a little worse than younger people.[5] One reason is numeracy drops in old age, often alongside problems like Alzheimer’s and age-related cognitive decline. Older people tend to resort to intuition, and younger people tend to think analytically. Both tendencies might result from younger people being engaged in a competitive world and older people retiring and relaxing from cognitive work.
Students from some of the top universities[6] in the world score about 1.5 out of 3. With many students scoring 0. So, there is no clear “intellectual standard” that education alone gives for the CRT. But, a high score in the CRT does predict a high score in many competitive exams like the ACT and SAT.
A large study on 118 small CRT studies on 44000+ people across 21 countries[7] shows that students typically perform better than non-students (highly likely this is because of the critical thinking mindset students need to regularly use) and men perform slightly better than women (highly likely due to cultural differences in how much men are encouraged to think critically and take math/science-related roles in society).
Daniel Kahneman, the Nobel prize winner, and the author of thinking fast and slow[8], describes 2 types of thinking: Type 1 thinking (aka system 1 thinking) and Type 2 thinking (aka system 2 thinking). Type 1 is fast, automatic, and intuitive. Type 2 is slow, deliberate, and analytical. Those who score low in the CRT use Type 1 thinking more and those who score high use Type 2 thinking more… at least during the test.
People who score high on a personality trait called “need for cognition” may score higher on the CRT because that personality trait is a default tendency to think elaborately and deeply about a problem. They tend to analyze more than someone who scores low on the need for cognition trait.
Practice these
Not happy with your score? Try taking this non-numerical version of it to engage your brain in more analytical and fun thinking. I won’t give the answers, so I’ll leave you with it! Some of these might not be culturally relevant to you. Still, give it a shot. And test your friends!
- If you are running a race and you pass the person in second place, what place are you in?
- Emily’s father has three daughters. The first two are named Monday and Tuesday. What is the third daughter’s name?
- A farmer has five sheep, all but three run away. How many are left?
- If there are three apples and you take away two, how many do you have?
- What do cows drink?
- What hatches from a butterfly egg?
- What weighs more, a pound of rocks or a pound of feathers?
- Who makes Christmas presents at the North Pole?
- Anna is playing four square with her three friends: Eeny, Meeny, and Miny. Who is the fourth player?
Sources
[2]: https://peerj.com/articles/2395/
[3]: https://onlinelibrary.wiley.com/doi/10.1111/cogs.12966
[4]: https://www.sciencedirect.com/science/article/abs/pii/S0191886916307851
[5]: https://www.tandfonline.com/doi/abs/10.1080/0361073X.2017.1398508
[6]: https://www.aeaweb.org/articles?id=10.1257/089533005775196732
[7]: https://www.sciencedirect.com/science/article/abs/pii/S2214804319301569
[8]: https://amzn.to/3qiunIj
Hey! Thank you for reading; hope you enjoyed the article. I run Cognition Today to paint a holistic picture of psychology. My content here is referenced and featured in NY Times, Forbes, CNET, Entrepreneur, Lifehacker, 10-15 books, academic courses, and research papers.
I’m a full-time psychology blogger, part-time Edtech and cyberpsychology consultant, guitar trainer, and also overtime impostor. I’ve studied at NIMHANS Bangalore (positive psychology), Savitribai Phule Pune University (clinical psychology), and IIM Ahmedabad (marketing psychology).
I’m based in Pune, India. Love sci-fi, horror media; Love rock, metal, synthwave, and pop music; can’t whistle; can play 2 guitars at a time.
Help me run this site with a donation :)
