Explainer
Comment
Economics
Leading
Politics
Wisdom
5 min read

When someone makes a claim, ask yourself these questions

How stories, statistics, and studies exploit our biases.

Alex is a professor of finance, and an expert in the use and misuse of data and evidence.

A member of an audience makes a point while gesturing.
On the other hand...
Antenna on Unsplash.

“Check the facts.”  

“Examine the evidence.”  

“Correlation is not causation.”  

We’ve heard these phrases enough times that they should be in our DNA. If true, misinformation would never get out of the starting block. But countless examples abound of misinformation spreading like wildfire. 

This is because our internal, often subconscious, biases cause us to accept incorrect statements at face value. Nobel Laureate Daniel Kahneman refers to our rational, slow thought process – which has mastered the above three phrases – as System 2, and our impulsive, fast thought process – distorted by our biases – as System 1. In the cold light of day, we know that we shouldn’t take claims at face value, but when our System 1 is in overdrive, the red mist of anger clouds our vision. 

Confirmation bias 

One culprit is confirmation bias – the temptation to accept evidence uncritically if it confirms what we’d like to be true, and to reject a claim out of hand if it clashes with our worldview. Importantly, these biases can be subtle; they’re not limited to topics such as immigration or gun control where emotions run high. It’s widely claimed that breastfeeding increases child IQ, even though correlation is not causation because parental factors drive both. But, because many of us would trust natural breastmilk over the artificial formula of a giant corporation, we lap this claim up. 

Confirmation bias is hard to shake. In a study, three neuroscientists took students with liberal political views and hooked them up to a functional magnetic resonance imaging scanner. The researchers read out statements the participants previously said they agreed with, then gave contradictory evidence and measured the students’ brain activity. There was no effect when non-political claims were challenged, but countering political positions triggered their amygdala. That’s the same part of the brain that’s activated when a tiger attacks you, inducing a ‘fight-or-flight’ response. The amygdala drives our System 1, and drowns out the prefrontal cortex which operates our System 2. 

Confirmation bias looms large for issues where we have a pre-existing opinion. But for many topics, we have no prior view. If there’s nothing to confirm, there’s no confirmation bias, so we’d hope we can approach these issues with a clear head. 

Black-and-white thinking 

Unfortunately, another bias can kick in: black-and-white thinking. This bias means that we view the world in binary terms. Something is either always good or always bad, with no shades of grey. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

The bestselling weight-loss book in history, Dr Atkins’ New Diet Revolution, benefited from this bias. Before Atkins, people may not have had strong views on whether carbs were good or bad. But as long as they think it has to be one or the other, with no middle ground, they’ll latch onto a one-way recommendation. That’s what the Atkins diet did. It had one rule: Avoid all carbs. Not just refined sugar, not just simple carbs, but all carbs. You can decide whether to eat something by looking at the “Carbohydrate” line on the nutrition label, without worrying whether the carbs are complex or simple, natural or processed. This simple rule played into black-and-white thinking and made it easy to follow. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

Overcoming Our biases 

So, what do we do about it? The first step is to recognize our own biases. If a statement sparks our emotions and we’re raring to share or trash it, or if it’s extreme and gives a one-size-fit-all prescription, we need to proceed with caution. 

The second step is to ask questions, particularly if it’s a claim we’re eager to accept. One is to “consider the opposite”. If a study had reached the opposite conclusion, what holes would you poke in it? Then, ask yourself whether these concerns still apply even though it gives you the results you want. 

Take the plethora of studies claiming that sustainability improves company performance. What if a paper had found that sustainability worsens performance? Sustainability supporters would throw up a host of objections. First, how did the researchers actually measure sustainability? Was it a company’s sustainability claims rather than its actual delivery? Second, how large a sample did they analyse? If it was a handful of firms over just one year, the underperformance could be due to randomness; there’s not enough data to draw strong conclusions. Third, is it causation or just correlation? Perhaps high sustainability doesn’t cause low performance, but something else, such as heavy regulation, drives both. Now that you’ve opened your eyes to potential problems, ask yourselves if they plague the study you’re eager to trumpet. 

A second question is to “consider the authors”. Think about who wrote the study and what their incentives are to make the claim that they did. Many reports are produced by organizations whose goal is advocacy rather than scientific inquiry. Ask “would the authors have published the paper if it had found the opposite result?” — if not, they may have cherry-picked their data or methodology. 

In addition to bias, another key attribute is the authors’ expertise in conducting scientific research. Leading CEOs and investors have substantial experience, and there’s nobody more qualified to write an account of the companies they’ve run or the investments they’ve made. However, some move beyond telling war stories to proclaiming a universal set of rules for success – but without scientific research we don’t know whether these principles work in general. A simple question is “If the same study was written by the same authors, with the same credentials, but found the opposite results, would you still believe it?” 

Today, anyone can make a claim, start a conspiracy theory or post a statistic. If people want it to be true it will go viral. But we have the tools to combat it. We know how to show discernment, ask questions and conduct due diligence if we don’t like a finding. The trick is to tame our biases and exercise the same scrutiny when we see something we’re raring to accept. 

 

This article is adapted from May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – and What We Can Do About It
(Penguin Random House, 2024)
Reproduced by kind permission of the author.

Celebrate our 2nd birthday!

Since Spring 2023, our readers have enjoyed over 1,000 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?

Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief

Article
Character
Comment
Friendship
Virtues
6 min read

Guiding’s new badges don’t deserve this media criticism

Encouraging interests builds character now and for tomorrow

Paula Duncan is a PhD candidate at the University of Aberdeen, researching OCD and faith.

Two Brownies point out their badges.
Girl Guiding.

As a Brownie leader, what I love about Girlguiding is the perfect blend of tradition and modernity. Lots of people will have memories of belonging to Brownies or Guides. In conversation, it’s often the uniforms that people mention first – for ex-Brownies it’s always the brown dresses, or the iconic yellow sweatshirt which has now been relaunched for adults as a retro range. I will admit to buying one almost immediately and have loved wearing my tribute to the Jeff Banks design era of uniform that I loved so much as a child.  

For those unfamiliar with the organisation, Girlguiding began in 1909 with a small group of girls who joined a Scout rally and bravely asked that there be ‘something for the girls’ too. Today, Girlguiding has four youth sections: Rainbows (aged 4-7), Brownies (7-10), Guides (10-14), and Rangers (14-18).  

I’m particularly excited to go back to our weekly meetings after the school holidays because a new range of interest badges have just been launched for all sections – 78 in total! These badges are designed so that girls can choose which activities they’d like to pursue – either a new skill or hobby or learning something new. Each badge fits into one of our six distinct programme areas: Be Well, Express Myself, Skills for My Future, Take Action, Know Myself, and Have Adventures.  

A lot of the publicity surrounding the badges has been really positive – praising the organisation for representing the interests of young people. The badges were designed, after all, with the input of members and with extensive testing by groups across the UK, and feedback from 11,000 girls about the new badge offerings. My Brownie unit enjoyed being part of the testing pool for the ‘Passions’ badge, which encouraged them to think more deeply about the hobbies they have and try some new pursuits. Reading the final version of the badge when it was released was brilliant because I could really see the impact of the Brownies’ feedback.  

I’ve been disappointed to see the organisation’s new badges be subject to some heavy media criticism and that headlines have been tailored to mock, rather than share the relevant and interesting content that the badges actually offer. The Telegraph used the current tensions around gender identity to create a clickbait title: “Girlguiding ‘hostess’ badge gets gender-neutral makeover” which, inevitably, caused a flurry of (largely unpleasant) Facebook comments about identity politics. While the article itself does well to describe some of the new badge offerings, leading with a controversial heading detracts from the truth that the previous ‘hostess’ badge was discontinued as part of the programme refresh and that member feedback brought about a return of a similar offering.  

Victoria Richards writes in The Independent that: ‘Girlguiding is […] modernising the way it interacts with young people. Speaking their language. After all, what use is an organisation for girls if nobody wants to join it?’  

This does not seem to be the opinion of Lucy Mangan who writes a particularly scathing indictment of the movement in general and in a large part advocates for teaching children how to use a gun and cut down trees. Obviously unfamiliar with the Girlguiding programme, she misses that our new interest badges are just that – part of a programme that encourages girls to pursue their own interests and often to try something new! “Interest” simply means that we allow girls to choose for themselves what they are interested in learning more about. We remain dedicated to teaching core life skills (things like first aid, I’d argue, rather than using a gun) as part of the Skills Builder set of badges that progress through each section.  

An opinion piece in The Express says: ‘The new Girl Guide badges are so easy, it’s like awarding a gold star for blinking’… This simply isn’t true; even the youngest members are encouraged to try something new or improve on a skill. It’s strange that none of these critical articles come from current members or leaders. Sometimes we do offer badges for participation at an event or marking on occasion but part of the joy of those is collecting them. I love looking for a badge patch when I’m on holiday and adding them to my camp blanket. There are some fantastic blankets out there – with badges from people’s travels, awards they’ve earned, and badges they’ve swapped with others they’ve met at events.  

Why shouldn’t we be encouraging our youngest members to have courage?

Most articles – including the BBC – have chosen to lead articles with the range of badges that fall under ‘Be Well’ or ‘Know Myself’ and are therefore designed to help young people learn valuable skills in self-care and identifying the things that are important to them. They are, by nature, designed to help young people learn more about themselves and how they interact with the world.  

This leads readers to believe that all badges are ‘self-centred’ as critics like Mangan have written. Why shouldn’t we be encouraging our youngest members to have courage? Girlguiding’s research has shown that 50 per cent of girls surveyed felt anxious about their future in 2024. Showing girls how to manage feelings of worry from a young age can only be a good thing. Our Rainbows are encouraged to “share the laughter with someone else.” Brownie Guides are working on their friendship badge are asked to “spend time with a new or old friend”.  

Dig a little further and look to the other interest badges and there can be no doubt that the programme Girlguiding offers is important and helps young people to challenge themselves and build new skills. The activities offered are relevant to the concerns of young people and the betterment of their communities. Rangers are supported to learn more about voting and how to ensure their voice is heard. In a world with fast fashion and markets like Temu dominating digital spaces, Guides can find out how to make informed decisions as part of their Conscious Consumer badge. 

Brownies can learn new languages or work towards their Mechanic badge. Even the youngest girls have the chance to learn about key principles of architecture in their Construction badge – the full syllabus of which is online. For critics who say that traditional skills like knot tying, sewing, or semaphore are forgotten by Girlguiding, all of these can be found in our Skills Builders and unit meeting activities. It can be so easy to be critical of something new or something that is changing without doing any further research to find out whether the headlines are accurate. Clickbait headlines are designed to drive up traffic and revenue, but something of the truth is lost in the process.  

I am incredibly proud to be part of an organisation that offers a rich, varied programme to young people. It has helped me grow from an anxious primary school child to a leader who can recognise the value of teaching life skills that are relevant to our time and place. Right now, girls need to have the resources and support to thrive in both the tangible world, and in an increasingly digital world.  

For those disheartened by the reception of the new badges, it’s worth looking to smaller news outlets. Those who have taken the time to garner responses from members and young people are much more encouraging and help to celebrate an exciting new step in our programme. I hope that those who can see the value of the programme offered to girls through Girlguiding can use this opportunity to find out more. If anything in this article has resonated with you, please do check out volunteer opportunities. 

From someone who likes to incorporate things like Star Trek into my academic work, why shouldn’t we be able to offer a badge about Fandoms? Why shouldn’t we celebrate the things that we enjoy? There really is something for every girl, even if the loudest voices in Facebook comment sections don’t make that clear. I’m incredibly proud to be part of an organisation that brings out the best in young people (and in me as an adult leader!).  

Support Seen & Unseen

Since Spring 2023, our readers have enjoyed over 1,500 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work? 

Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief