Explainer
Comment
Economics
Leading
Politics
Wisdom
5 min read

When someone makes a claim, ask yourself these questions

How stories, statistics, and studies exploit our biases.

Alex is a professor of finance, and an expert in the use and misuse of data and evidence.

A member of an audience makes a point while gesturing.
On the other hand...
Antenna on Unsplash.

“Check the facts.”  

“Examine the evidence.”  

“Correlation is not causation.”  

We’ve heard these phrases enough times that they should be in our DNA. If true, misinformation would never get out of the starting block. But countless examples abound of misinformation spreading like wildfire. 

This is because our internal, often subconscious, biases cause us to accept incorrect statements at face value. Nobel Laureate Daniel Kahneman refers to our rational, slow thought process – which has mastered the above three phrases – as System 2, and our impulsive, fast thought process – distorted by our biases – as System 1. In the cold light of day, we know that we shouldn’t take claims at face value, but when our System 1 is in overdrive, the red mist of anger clouds our vision. 

Confirmation bias 

One culprit is confirmation bias – the temptation to accept evidence uncritically if it confirms what we’d like to be true, and to reject a claim out of hand if it clashes with our worldview. Importantly, these biases can be subtle; they’re not limited to topics such as immigration or gun control where emotions run high. It’s widely claimed that breastfeeding increases child IQ, even though correlation is not causation because parental factors drive both. But, because many of us would trust natural breastmilk over the artificial formula of a giant corporation, we lap this claim up. 

Confirmation bias is hard to shake. In a study, three neuroscientists took students with liberal political views and hooked them up to a functional magnetic resonance imaging scanner. The researchers read out statements the participants previously said they agreed with, then gave contradictory evidence and measured the students’ brain activity. There was no effect when non-political claims were challenged, but countering political positions triggered their amygdala. That’s the same part of the brain that’s activated when a tiger attacks you, inducing a ‘fight-or-flight’ response. The amygdala drives our System 1, and drowns out the prefrontal cortex which operates our System 2. 

Confirmation bias looms large for issues where we have a pre-existing opinion. But for many topics, we have no prior view. If there’s nothing to confirm, there’s no confirmation bias, so we’d hope we can approach these issues with a clear head. 

Black-and-white thinking 

Unfortunately, another bias can kick in: black-and-white thinking. This bias means that we view the world in binary terms. Something is either always good or always bad, with no shades of grey. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

The bestselling weight-loss book in history, Dr Atkins’ New Diet Revolution, benefited from this bias. Before Atkins, people may not have had strong views on whether carbs were good or bad. But as long as they think it has to be one or the other, with no middle ground, they’ll latch onto a one-way recommendation. That’s what the Atkins diet did. It had one rule: Avoid all carbs. Not just refined sugar, not just simple carbs, but all carbs. You can decide whether to eat something by looking at the “Carbohydrate” line on the nutrition label, without worrying whether the carbs are complex or simple, natural or processed. This simple rule played into black-and-white thinking and made it easy to follow. 

To pen a bestseller, Atkins didn’t need to be right. He just needed to be extreme. 

Overcoming Our biases 

So, what do we do about it? The first step is to recognize our own biases. If a statement sparks our emotions and we’re raring to share or trash it, or if it’s extreme and gives a one-size-fit-all prescription, we need to proceed with caution. 

The second step is to ask questions, particularly if it’s a claim we’re eager to accept. One is to “consider the opposite”. If a study had reached the opposite conclusion, what holes would you poke in it? Then, ask yourself whether these concerns still apply even though it gives you the results you want. 

Take the plethora of studies claiming that sustainability improves company performance. What if a paper had found that sustainability worsens performance? Sustainability supporters would throw up a host of objections. First, how did the researchers actually measure sustainability? Was it a company’s sustainability claims rather than its actual delivery? Second, how large a sample did they analyse? If it was a handful of firms over just one year, the underperformance could be due to randomness; there’s not enough data to draw strong conclusions. Third, is it causation or just correlation? Perhaps high sustainability doesn’t cause low performance, but something else, such as heavy regulation, drives both. Now that you’ve opened your eyes to potential problems, ask yourselves if they plague the study you’re eager to trumpet. 

A second question is to “consider the authors”. Think about who wrote the study and what their incentives are to make the claim that they did. Many reports are produced by organizations whose goal is advocacy rather than scientific inquiry. Ask “would the authors have published the paper if it had found the opposite result?” — if not, they may have cherry-picked their data or methodology. 

In addition to bias, another key attribute is the authors’ expertise in conducting scientific research. Leading CEOs and investors have substantial experience, and there’s nobody more qualified to write an account of the companies they’ve run or the investments they’ve made. However, some move beyond telling war stories to proclaiming a universal set of rules for success – but without scientific research we don’t know whether these principles work in general. A simple question is “If the same study was written by the same authors, with the same credentials, but found the opposite results, would you still believe it?” 

Today, anyone can make a claim, start a conspiracy theory or post a statistic. If people want it to be true it will go viral. But we have the tools to combat it. We know how to show discernment, ask questions and conduct due diligence if we don’t like a finding. The trick is to tame our biases and exercise the same scrutiny when we see something we’re raring to accept. 

 

This article is adapted from May Contain Lies: How Stories, Statistics, and Studies Exploit Our Biases – and What We Can Do About It
(Penguin Random House, 2024)
Reproduced by kind permission of the author.

Celebrate our 2nd birthday!

Since Spring 2023, our readers have enjoyed over 1,000 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?

Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief

Essay
Comment
Community
Identity
Politics
8 min read

The country needs fixing, here’s where to start

Turning back the clock, closing the gates, and putting up more flags, is not the answer
A commemorative blue plaque on a a wall is smashed to pieces
Julian Hochgesang on Unsplash.

This is the third in a series of articles clustered around the ideas of constitutionalism, Christianity and national identity. The first article set out the case for a new written constitution in order to protect against the authoritarian reactionary populism that threatens to undermine democratic norms and institutions. The second article made a distinction between England’s ‘metaphysical’ constitution and its ‘mechanical’ constitution – arguing that the former is in need of restoration, and the latter in need of thorough reform.  

This article concludes with a plea for peace and unity. There is not much, in this increasingly febrile and polarised political climate, that left and right agree upon. Who is responsible for the mess we are in: the Romanians or the Etonians? Against whom should we direct our ire: the people in small boats in the Channel, or the people in big yachts in the Cayman Islands? Was Thatcher a hero or a villain? Was the purity of Brexit bliss betrayed by scheming Remoaners, or was leaving the European Union always going to be a disaster? 

What we can agree upon, however, is that something is deeply broken. While we might disagree on the causes and the solutions, the unavoidable evidence of brokenness is before our eyes. In every area of daily life, things are – to use the most fitting vernacular expression – ‘a bit pants’. Perhaps things are not utterly dire, in the way that much of somewhere like Sudan or Burma is utterly dire, but they are nevertheless far from the standard that one might reasonably expect from the rich, first world, country we still claim to be.  

Park life 

Go, for example, to your local park, if there is one. Observe the broken glass, the graffiti, the used condoms, the discarded drug paraphernalia, the joyless air of bleak menace in a place that should be a happy sanctuary for children and families. If that description does not match your experience, perhaps you are one of the lucky ones, who lives in a good area – but many of your fellows, in dull decaying provincial towns, are much less fortunate.  

Parks are just one manifestation of a land in the doldrums. One might just as well point to the fact that since privatisation water companies have not built any new reservoirs, or to uncollected rubbish piling up in the streets of Birmingham, or to the difficulty of getting an NHS dental appointment. Everything is tired, run down, threadbare, falling apart.  

Those in charge, nationally and locally, are stretched between the irreconcilable demands of expanding needs and tight budgets. Engulfed by short-term crisis-management, they lack the ability to look up, grab the situation by the horns, and bring about the fundamental, structural and systematic change that is needed to actually fix things.  

The decay is evident, too, in society at large. Employment, for those who can get it, is characterised by low wages and precarity. Housing costs are absurd. People at all levels have become exhausted, demoralised, bored and lacklustre, locked in an ‘overwhelm paralysis’. The public mood has become despondent, cynical, ineffectually angry, but also frightened. Many are grimly hanging on, just going through the motions to the minimal extent necessary to endure the week, and afford the month. Even basic civility and politeness have worn thin.  

This is a far cry from a ‘Land of hope and glory’. Indeed, if one were to take honest stock of things, one might conclude that we live in a state where there is none righteous, and where all have sinned and fallen short of the glory of God.  

Cobwebbed constitution 

In making this sudden metaphysical turn – jumping from the decay, despondency and desperation evident around us, to the realm of the spirit – I do not wish for a moment to minimise the importance of such mundane human affairs as ideologies and policies. What government does and does not do, and what law allows or prohibits, matters. It should be no surprise that if governments leave undone those things which they ought to have done, and do those things which they ought not to have done, then there will be, at the end, ‘no health in us’.  

Still less would I wish to neglect the role of institutions and structures – and, ultimately, the constitution itself. The woes we experience, in terms of bad policy and poor execution, are largely the result of an ill-constituted state. The words of Tom Paine (a man ill-remembered by history, but one of the few Englishmen to have understood the centrality of constitutional matters) are as true today as when he penned them more than two centuries ago:  

‘For want of a constitution in England to restrain and regulate the wild impulse of power, many of the laws are irrational and tyrannical, and the administration of them vague and problematical.’  

When it comes to mending a country in decline, the first thing to do is to make sure the constitution is sound, and that the institutions of democracy and governance work as they should. Behind all the policy and governance failures, of both Conservatives and Labour, lies the fact that we are governed by the cobwebbed remains of a once mighty commercial imperial state, now hollowed out by neoliberalism, without any clear ethical principles to direct or sustain it. To expect good outcomes from such an ill-constituted state would be as absurd as expecting to gather figs from thorns, or grapes from briers. 

We might even put it in these terms: Every good constitution brings forth good government; but a corrupt constitution brings forth evil government. A good constitution cannot bring forth evil government, neither can a corrupt constitution bring forth good government. 

Constitutional renovation has therefore become a precondition for the restoration of the legitimacy, credibility, authority, and moral integrity of the state, as well as for the health, well-being, and prosperity of the people. This calls for quite a different project of national renewal from that offered by offered by the parties of the reactionary right. Simply turning back the clock, closing the gates, and putting up more flags, is not the answer.  

Governo largo 

The centrepiece of a national renewal project should be constitutional: to create a truly ‘public state’ – a democratic state founded upon, oriented towards, and capable of serving, the common good. Tend to that tree, water its constitutional roots, and the fruits will follow.  

Again, Paine tells us what the fruits of that good tree are, and therefore how to recognise when the constitutional tree is healthy:  

‘When it shall be said in any country in the world, my poor are happy; neither ignorance nor distress is to be found among them; my jails are empty of prisoners, my streets of beggars; the aged are not in want, the taxes are not oppressive; the rational world is my friend, because I am the friend of its happiness: when these things can be said, then may that country boast its constitution and its government.’ 

This is little more than a restatement of the basic Aristotelean distinction between good and bad government. Good government (the well-constituted state, or ‘polity’) governs in the public interest, for the common good, while all forms of bad government – tyranny, oligarchy and populism – govern in the private interests of the rulers, perverting public power for personal gain. 

The renaissance Italian statesman, Francesco Guicciardini, highlighted this distinction in clearer, more binary terms. He contrasted the ‘governo largo’ with the ‘governo stretto’. A governo largo is a wide, open, broad-based government, in which power is broadly shared and publicly accountable, so that public life is centred upon public needs. It is system of government not only by and of the people, but also for the people. ‘Governo stretto’, in contrast, is a narrow, restricted, closed, private, self-seeking, public-ignoring state. 

The first attempt at constituting a ‘governo largo’ in England was made during the Civil Wars, with the ‘Agreement of the People’. This went through several drafts between 1647 and 1649. The title was well chosen. Real, working, constitutions are produced through a process of discussion and negotiation – ‘arguing and bargaining’ – that enables a broadly acceptable constitutional settlement to be reached. The constitution expresses what been agreed, amongst the people or their representatives, as the common foundation of the state.  

Reaching such an agreement today, in a society that has become as polarised and divided as ours, will not be easy. It is nevertheless necessary. In order to establish a state that serves the common good, we must have some agreed foundations, ground-rules, shared principles, upon which a general consensus exists. This alone can provide the basis for an inclusive, publicly-oriented, ‘governo largo’.  

This is not a radical innovation. Almost every country which has become independent from the British Empire has adopted a democratic constitution as its supreme and fundamental law. In some cases – in India in 1950, South Africa in 1996, and Kenya in 2010 – a serious attempt was made to establish an inclusive ‘governo largo’ constitution. In so doing, they sought to heal deep divisions, to reach a broadly acceptable settlement, and thereby to make good government – and with it socio-economic development – at least possible. 

Perhaps we think we are better than all that, beyond such constitutional trifles. Yet, the fact remains that our politics today – and our society today – look much more like those of India, South Africa and Kenya than, say, like those of 1950s England. Either we find ways to dwell together in unity, or we face the kind of civil breakdown which the ancients referred to as ‘stasis’, in which all notions of the common good and the public interest are abandoned in partisan, factional, sectarian or ethnic conflict.  

Here then, we must return to matters of the spirit. A good constitution is necessary, but the best constitution cannot save us. A constitution might call us to liberty – to that political freedom which enables us, as responsible citizens, to exercise care for common things, through systems of representative and responsible ‘public government’, but that is not enough, unless we also cultivate the qualities of character to use liberty well and wisely.  

Saint Paul enjoins us not to use liberty ‘for an occasion to the flesh’ – that is, to seek our own, selfish, corrupt or partisan ends. He warns us perils of stasis: ‘But if ye bite and devour one another, take heed that ye be not consumed one of another.’  He also points to that one solution by which the degeneracy of the state, and the corruption of the constitution, might ultimately be overcome: ‘all the law is fulfilled in one word, even in this; Thou shalt love thy neighbour as thyself.’ 

In other words, if we wish to seek the common good, to be well governed, to live in peace and unity, with freedom and justice, then we have to learn to love one another. Civic and political regeneration cannot ultimately be separated from regeneration of our souls.  

Support Seen & Unseen

Since Spring 2023, our readers have enjoyed over 1,500 articles. All for free. 
This is made possible through the generosity of our amazing community of supporters.

If you enjoy Seen & Unseen, would you consider making a gift towards our work?
 
Do so by joining Behind The Seen. Alongside other benefits, you’ll receive an extra fortnightly email from me sharing my reading and reflections on the ideas that are shaping our times.

Graham Tomlin
Editor-in-Chief