Explainer
Comment
Economics
Leading
Politics
Wisdom
5 min read

When someone makes a claim, ask yourself these questions

How stories, statistics, and studies exploit our biases.

Alex is a professor of finance, and an expert in the use and misuse of data and evidence.

A member of an audience makes a point while gesturing.
On the other hand...
Antenna on Unsplash.

“Check the facts.”  

“Examine the evidence.”  

“Correlation is not causation.”  

We’ve heard these phrases enough times that they should be in our DNA. If true, misinformation would never get out of the starting block. But countless examples abound of misinformation spreading like wildfire. 

This is because our internal, often subconscious, biases cause us to accept incorrect statements at face value. Nobel Laureate Daniel Kahneman refers to our rational, slow thought process – which has mastered the above three phrases – as System 2, and our impulsive, fast thought process – distorted by our biases – as System 1. In the cold light of day, we know that we shouldn’t take claims at face value, but when our System 1 is in overdrive, the red mist of anger clouds our vision. 

Confirmation bias 

One culprit is confirmation bias – the temptation to accept evidence uncritically if it confirms what we’d like to be true, and to reject a claim out of hand if it clashes with our worldview. Importantly, these biases can be subtle; they’re not limited to topics such as immigration or gun control where emotions run high. It’s widely claimed that breastfeeding increases child IQ, even though correlation is not causation because parental factors drive both. But, because many of us would trust natural breastmilk over the artificial formula of a giant corporation, we lap this claim up. 

Confirmation bias is hard to shake. In a study, three neuroscientists took students with liberal political views and hooked them up to a functional magnetic resonance imaging scanner. The researchers read out statements the participants previously said they agreed with, then gave contradictory evidence and measured the students’ brain activity. There was no effect when non-political claims were challenged, but countering political positions triggered their amygdala. That’s the same part of the brain that’s activated when a tiger attacks you, inducing a ‘fight-or-flight’ response. The amygdala drives our System 1, and drowns out the prefrontal cortex which operates our System 2. 

Confirmation bias looms large for issues where we have a pre-existing opinion. But for many topics, we have no prior view. If there’s nothing to confirm, there’s no confirmation bias, so we’d hope we can approach these issues with a clear head. 

Black-and-white thinking 

Unfortunately, another bias can kick in: black-and-white thinking. This bias means that we view the world in binary terms. Something is either always good or always bad, with no shades of grey. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

The bestselling weight-loss book in history, Dr Atkins’ New Diet Revolution, benefited from this bias. Before Atkins, people may not have had strong views on whether carbs were good or bad. But as long as they think it has to be one or the other, with no middle ground, they’ll latch onto a one-way recommendation. That’s what the Atkins diet did. It had one rule: Avoid all carbs. Not just refined sugar, not just simple carbs, but all carbs. You can decide whether to eat something by looking at the “Carbohydrate” line on the nutrition label, without worrying whether the carbs are complex or simple, natural or processed. This simple rule played into black-and-white thinking and made it easy to follow. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

Overcoming Our biases 

So, what do we do about it? The first step is to recognize our own biases. If a statement sparks our emotions and we’re raring to share or trash it, or if it’s extreme and gives a one-size-fit-all prescription, we need to proceed with caution. 

The second step is to ask questions, particularly if it’s a claim we’re eager to accept. One is to “consider the opposite”. If a study had reached the opposite conclusion, what holes would you poke in it? Then, ask yourself whether these concerns still apply even though it gives you the results you want. 

Take the plethora of studies claiming that sustainability improves company performance. What if a paper had found that sustainability worsens performance? Sustainability supporters would throw up a host of objections. First, how did the researchers actually measure sustainability? Was it a company’s sustainability claims rather than its actual delivery? Second, how large a sample did they analyse? If it was a handful of firms over just one year, the underperformance could be due to randomness; there’s not enough data to draw strong conclusions. Third, is it causation or just correlation? Perhaps high sustainability doesn’t cause low performance, but something else, such as heavy regulation, drives both. Now that you’ve opened your eyes to potential problems, ask yourselves if they plague the study you’re eager to trumpet. 

A second question is to “consider the authors”. Think about who wrote the study and what their incentives are to make the claim that they did. Many reports are produced by organizations whose goal is advocacy rather than scientific inquiry. Ask “would the authors have published the paper if it had found the opposite result?” — if not, they may have cherry-picked their data or methodology. 

In addition to bias, another key attribute is the authors’ expertise in conducting scientific research. Leading CEOs and investors have substantial experience, and there’s nobody more qualified to write an account of the companies they’ve run or the investments they’ve made. However, some move beyond telling war stories to proclaiming a universal set of rules for success – but without scientific research we don’t know whether these principles work in general. A simple question is “If the same study was written by the same authors, with the same credentials, but found the opposite results, would you still believe it?” 

Today, anyone can make a claim, start a conspiracy theory or post a statistic. If people want it to be true it will go viral. But we have the tools to combat it. We know how to show discernment, ask questions and conduct due diligence if we don’t like a finding. The trick is to tame our biases and exercise the same scrutiny when we see something we’re raring to accept. 

 

This article is adapted from May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – and What We Can Do About It
(Penguin Random House, 2024)
Reproduced by kind permission of the author.

Celebrate our 2nd birthday!

Since Spring 2023, our readers have enjoyed over 1,000 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?

Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief

Article
Culture
Digital
Film & TV
Work
7 min read

What my film about the prodigal son really means

Our relentless focus on productivity devalues the things that make us human

Emily is designer and animator at the Theos think tank.

An animated man runs through a jungle.
In Sync with the Sun.
Theos.

Watch now

In his 2021 book 4,000 weeks: Time Management for Mortals, Oliver Burkeman observes that an obsession with productivity doesn’t give us more control over our lives, ‘instead, life accelerates, and everyone grows more impatient. It’s somehow vastly more aggravating to wait two minutes for the microwave than two hours for the oven - or ten seconds for a slow-loading web page versus three days to receive the same information by post.’ 

With technologies like artificial intelligence rapidly accelerating our lives, this constant demand to squeeze more into our time is not only limited to the mundane tasks that we have to do and wish we didn’t. It seeps into what we want to do and indeed must do in order to flourish: creating art, spending time in community, and caring for others. The problem is that these things cannot be measured in productivity metrics because they inherently do not function in that way. How do you measure how ‘productive’ a conversation is? Or a work of art? Artists such as Vincent Van Gogh or Emily Dickinson didn’t see their influence in their own lifetime. 

The more we measure our lives in productivity metrics, the more we devalue the things that make us human, ultimately making our lives and the world around us increasingly artificial. This is the basis of my recent film, In Sync with the Sun, which is a short animation about the rhythms of activity and rest that are written into our world, and what happens when an obsession with productivity takes over.  

I wrote the initial script for the film after a period of burnout. I was fully in the “make the most of every second” mindset, which left me feeling exhausted and confused about where my value resides. In response, I began researching the sleep-cycles of various animals and I was liberated by surprising details such as the fact that lions, which we see as mighty and majestic animals, sleep for around 21 hours a day. Even creatures like jellyfish, which don’t even have brains as far as we know, still have cycles of rest. Every living thing thrives in these rhythms of activity and rest, even down to plants and minuscule organisms. Our whole world is built on this pattern, in sync with the sun. Yet for us humans, our rhythms have been broken by technology, leaving us confused about our limitations and what we should do with our short lives.  

The film begins in nature, deep in the jungle where some leopards are sleeping. But the tranquility is abruptly interrupted by the voice-over declaring, “the war against sleep began when artificial broke into the night.” Brilliant white light breaks up the deep blues and purples on screen, until the screen is filled with blinding white. I wanted it to feel like that moment you peer at your phone in the middle of the night - the pain of your pupils trying to adjust. If you think about it, for 99.9 per cent of human history, our eyes would have never had to do that - until now.  

Artificial light wasn’t powerful enough to change that. Instead, it’s given us an unquenchable guilt about how we use our time. 

With his invention of the light bulb, Thomas Edison was determined to banish the night, and the limitations it enforced on us. Edison was known for being fiercely obsessed with productivity and, as a result, was an anti-sleep warrior who believed,

“There is really no reason why men should go to bed at all.”

As someone living a century on, I find it baffling to imagine that humans should eradicate sleep entirely. Perhaps because just 100 years later we are seeing the results that sleep-loss and over-working can have on our physical health and wellbeing. Maybe we cannot supersede nature after all, since we are an embedded part of it. It seems that “Sabbath" rest is written into our world and into our humanity. Artificial light wasn’t powerful enough to change that. Instead, it’s given us an unquenchable guilt about how we use our time. Now we decide when the day ends, so whoever can rest the least wins. 

The battle is still raging; incandescent bulbs only set aflame that root desire to be increasingly productive. The hamster wheel is spinning uncontrollably, and we must keep up. So, what do we do? The attempt to remove the limitations outside of us has revealed that they are in fact inside of us too. Therefore, the only way to keep up is to remove the human from the hamster wheel altogether. The failure of artificial light leads to the birth of artificial minds.  

 As a creative, this is what frustrates me most about artificial intelligence; that it is mostly being driven by this quest to bring everything under the reign of productivity. It goes without saying that this is greatly needed in some areas of society. Just like artificial light, it can and will do a lot of good in the world. However, when the obsession with productivity is prioritised over human flourishing, that’s when we know there is a big problem with how we view our lives.  

Thinking back to the examples of Van Gogh and Emily Dickinson; what is lost when we don’t allow space for artists, carers, mothers, or any skilled role that requires an element of patience? For me personally, I can’t force creative inspiration, instead it comes at me, often at times when I’m not looking for it. Similarly, sometimes that inspiration leads directly to an instant idea, but most often it’s a vague idea I jot down to which later life experiences and opportunities then build onto, forming it into something bigger and more in-depth. This could be compared to a role or situation that requires relationship building. Sometimes there are moments of instant bonding and “productive” progress in relationships, but it’s often more complex where external experiences or changes, which are outside of our control, may unexpectedly deepen understanding between people after long periods of frustration. 

In my animation, I used the metaphor of a butterfly to illustrate this sentiment. After the character realises he is not made for a life of relentless productivity, he steps out of the black and white skyscraper into the lush wilderness. A butterfly lands on his productivity badge and the voice over says, “You’re not a machine.” I imagine the Creator saying this to the loved creation. Creatures like butterflies seem completely unproductive to our human standards. They take weeks to form in the chrysalis and exist in the world for less time than that. Yet they are a source of wonder and beauty for anyone who has the privilege of seeing one up close. A reminder that nature is not in a rush. Where AI is concerned, however, speed and profit are the focus of desire. But looking at the world around us - that we are a part of - it’s clear that not everything can or should be valued by these limiting metrics alone. 

The overarching narrative of In Sync with the Sun is loosely inspired by the biblical story of the prodigal son. The main character has travelled far away from his home in pursuit of success, and he eventually realises that this master does not love him. At the end he comes home again, finding connection in community and in the good rhythm of productivity and rest that he came from. I wanted the film to address the issues that an unhealthy obsession with productivity can cause, and instead evoke a desire to accept and live more in sync with the boundaries and rhythms that are embedded in the natural world we are a part of.  

The film ends with the line, “The only thing that can stay awake is not awake at all.” In the midst of the changing world of AI, humans might be tempted to measure our productivity levels in comparison to these machines. However, technologies always raise the productivity bar higher and higher, and one day we need to accept that we simply aren’t going to be able to reach it. We don’t sit apart from nature like technology does, so let’s stop resenting that, and instead celebrate it. To quote Oliver Burkeman again,  

“the more you confront the facts of finitude instead - and work with them, rather than against them - the more productive, meaningful and joyful life becomes.” 

Support Seen & Unseen

Since Spring 2023, our readers have enjoyed over 1,500 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?
 
Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief