Explainer
Comment
Economics
Leading
Politics
Wisdom
5 min read

When someone makes a claim, ask yourself these questions

How stories, statistics, and studies exploit our biases.

Alex is a professor of finance, and an expert in the use and misuse of data and evidence.

A member of an audience makes a point while gesturing.
On the other hand...
Antenna on Unsplash.

“Check the facts.”  

“Examine the evidence.”  

“Correlation is not causation.”  

We’ve heard these phrases enough times that they should be in our DNA. If true, misinformation would never get out of the starting block. But countless examples abound of misinformation spreading like wildfire. 

This is because our internal, often subconscious, biases cause us to accept incorrect statements at face value. Nobel Laureate Daniel Kahneman refers to our rational, slow thought process – which has mastered the above three phrases – as System 2, and our impulsive, fast thought process – distorted by our biases – as System 1. In the cold light of day, we know that we shouldn’t take claims at face value, but when our System 1 is in overdrive, the red mist of anger clouds our vision. 

Confirmation bias 

One culprit is confirmation bias – the temptation to accept evidence uncritically if it confirms what we’d like to be true, and to reject a claim out of hand if it clashes with our worldview. Importantly, these biases can be subtle; they’re not limited to topics such as immigration or gun control where emotions run high. It’s widely claimed that breastfeeding increases child IQ, even though correlation is not causation because parental factors drive both. But, because many of us would trust natural breastmilk over the artificial formula of a giant corporation, we lap this claim up. 

Confirmation bias is hard to shake. In a study, three neuroscientists took students with liberal political views and hooked them up to a functional magnetic resonance imaging scanner. The researchers read out statements the participants previously said they agreed with, then gave contradictory evidence and measured the students’ brain activity. There was no effect when non-political claims were challenged, but countering political positions triggered their amygdala. That’s the same part of the brain that’s activated when a tiger attacks you, inducing a ‘fight-or-flight’ response. The amygdala drives our System 1, and drowns out the prefrontal cortex which operates our System 2. 

Confirmation bias looms large for issues where we have a pre-existing opinion. But for many topics, we have no prior view. If there’s nothing to confirm, there’s no confirmation bias, so we’d hope we can approach these issues with a clear head. 

Black-and-white thinking 

Unfortunately, another bias can kick in: black-and-white thinking. This bias means that we view the world in binary terms. Something is either always good or always bad, with no shades of grey. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

The bestselling weight-loss book in history, Dr Atkins’ New Diet Revolution, benefited from this bias. Before Atkins, people may not have had strong views on whether carbs were good or bad. But as long as they think it has to be one or the other, with no middle ground, they’ll latch onto a one-way recommendation. That’s what the Atkins diet did. It had one rule: Avoid all carbs. Not just refined sugar, not just simple carbs, but all carbs. You can decide whether to eat something by looking at the “Carbohydrate” line on the nutrition label, without worrying whether the carbs are complex or simple, natural or processed. This simple rule played into black-and-white thinking and made it easy to follow. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

Overcoming Our biases 

So, what do we do about it? The first step is to recognize our own biases. If a statement sparks our emotions and we’re raring to share or trash it, or if it’s extreme and gives a one-size-fit-all prescription, we need to proceed with caution. 

The second step is to ask questions, particularly if it’s a claim we’re eager to accept. One is to “consider the opposite”. If a study had reached the opposite conclusion, what holes would you poke in it? Then, ask yourself whether these concerns still apply even though it gives you the results you want. 

Take the plethora of studies claiming that sustainability improves company performance. What if a paper had found that sustainability worsens performance? Sustainability supporters would throw up a host of objections. First, how did the researchers actually measure sustainability? Was it a company’s sustainability claims rather than its actual delivery? Second, how large a sample did they analyse? If it was a handful of firms over just one year, the underperformance could be due to randomness; there’s not enough data to draw strong conclusions. Third, is it causation or just correlation? Perhaps high sustainability doesn’t cause low performance, but something else, such as heavy regulation, drives both. Now that you’ve opened your eyes to potential problems, ask yourselves if they plague the study you’re eager to trumpet. 

A second question is to “consider the authors”. Think about who wrote the study and what their incentives are to make the claim that they did. Many reports are produced by organizations whose goal is advocacy rather than scientific inquiry. Ask “would the authors have published the paper if it had found the opposite result?” — if not, they may have cherry-picked their data or methodology. 

In addition to bias, another key attribute is the authors’ expertise in conducting scientific research. Leading CEOs and investors have substantial experience, and there’s nobody more qualified to write an account of the companies they’ve run or the investments they’ve made. However, some move beyond telling war stories to proclaiming a universal set of rules for success – but without scientific research we don’t know whether these principles work in general. A simple question is “If the same study was written by the same authors, with the same credentials, but found the opposite results, would you still believe it?” 

Today, anyone can make a claim, start a conspiracy theory or post a statistic. If people want it to be true it will go viral. But we have the tools to combat it. We know how to show discernment, ask questions and conduct due diligence if we don’t like a finding. The trick is to tame our biases and exercise the same scrutiny when we see something we’re raring to accept. 

 

This article is adapted from May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – and What We Can Do About It
(Penguin Random House, 2024)
Reproduced by kind permission of the author.

Celebrate our 2nd birthday!

Since Spring 2023, our readers have enjoyed over 1,000 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?

Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief

Article
Comment
Mental Health
Politics
4 min read

Rachel Reeves’ tears: public life still mocks those who show anything but the positive

‘Mental health awareness’ is failing, our words are not matched by our actions

Rachael is an author and theology of mental health specialist. 

 

 

A woman sits and holds back a tear.
Rachel Reeves on the front bench.
Parliament TV.

It’s a bad day at work. Everyone is on high alert, and tempers are frayed. You have your own reasons for being extra ‘on edge’, but now isn’t the time to get into it because it’s the big weekly meeting and everyone is going to be there - worse still, the cameras are going to be there. Despite this, you take a deep breath and take your seat (which, although an honour, is regrettably in the front row).  

But as the fractious meeting begins, you feel the ache of impending tears at the back of your throat, and to your horror, your eyes fill. You do your best to wick them away, but you know they’ve been spotted when someone opposite announces how miserable you look. 

Many of us will have been in a similar, if probably less public, situation at some point in our careers when the emotions we stuff down in the name of professionalism spill out - but I doubt any of us will have done so in the House of Commons with cameras trained on every movement and a less than friendly crowd opposite.  

There have been countless articles already speculating about the reason for the tears of the Chancellor, Rachel Reeves, during Prime Minister’s Questions - but most seem devoid of sympathy or empathy, concerned only with the political implications, but not the person at the centre of this story.  

Our reaction to Rachel’s tears is an echo of the sentiment behind the Welfare Reform Bill, which seems to say that need is unacceptable and we should all be able to don that famously British ‘stiff upper lip’ and just get on with life.  

Regardless of what you think of the Welfare Reform Bill, the way it has been briefed and communicated has raised anxiety and fear amongst the disabled community (me included).  

The main message has been that too many people are receiving Personal Independence Payments (PIP) for mental illnesses such as anxiety and depression, with even the former Prime Minister Tony Blair telling people to ‘stop diagnosing themselves’ to combat out rising welfare bill - despite the fact that accessing PIP requires rigorous assessments and support from medical professionals. (It also has a 0.01% fraud rate and was designed to compensate people for the extra cost of being disabled which is estimated to be up to £1000 a month.) 

This tableau is emblematic of how ‘mental health awareness’ is failing in this country; our words are not matched by our actions. 

We know, 27 years after the first ‘Mental Health Awareness Week’, that mental health is important, that emotions are natural and valid - and yet we mock any leader who shows anything but positive emotions.  

We know that people suffer, are disabled by and killed by mental illnesses, and yet we seek to strip support from those who need it most, claiming that they are diagnosing themselves. 

We need a different approach, both to how we handle emotions in public life and the way we talk about those who need extra support due to their mental illnesses.  

Emotions aren’t bad - they help us connect, keep us away from danger and allow our bodies to release unbearable tension, as in the case of crying, whereby tears of pain are intricately designed to help us cope. The tears we shed when faced with chopping a pile of onions are chemically different to those that fall when we are grieving, angry or in pain. Tears of pain should inspire us to reach out to the one in pain with compassion not contempt.  

The way Jesus led 2,000 years ago shows us another way, both of leading and emoting.  

Jesus consistently welcomed those most in need; from healing the woman who had bled for twelve years, considered unclean and rejected by her community, to healing a paralysed man lowered through his roof by friends.  

And yet his ministry was not just one characterised by miracles and might, but demonstrated humility and humanity as he wept over the death of his friend Lazarus and allowed himself to be stripped of all strength as he hung on a cross made for criminals.  

The night before he died, he gathered his friends and through tears and blood-soaked sweat submitted to the Father in the most painful way, and I, like many others, draw comfort and strength from Jesus’ willingness to cry.  

As preacher Charles Haddon Spurgeon said, "A Jesus who never wept could never wipe away my tears."  

So perhaps rather than mock Rachel’s tears, they should cause us to rethink how we approach need and recognise none of us are immune.  

Perhaps, we may even join with Paul’s words in his letter to the Corinthians: “For when I am weak, then I am strong.” 

Support Seen & Unseen

Since Spring 2023, our readers have enjoyed over 1,500 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?
 
Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief