School spending hits record levels in England, claims a minister, while some schools close on Friday afternoons because of a lack of funds. China is the biggest environmental polluter in the world, Donald Trump once tweeted. But, per capita, Saudi Arabia emitted the largest tonnage of CO2 in 2018 and China was in 13th place.
What is it about statistics that make them a godsend to people intent on reinventing reality? They arrive with a veneer of scientific exactitude and probity – and the more exact they are, the greater their apparent accuracy. Better still, they convey an air of certitude and give us something to grasp on to in a turbulent world. Few of us are inclined to look deeper. But if we do, we’ll often find a mix of crude lies and sleights of hand – slippery definitions, manipulated percentages, cherrypicked comparisons or rough estimates presented as certainties.
Competent, honest statistics can illuminate essential truths. They can reveal social inequalities, indicate where resources or legislation should be directed, highlight dangers or help us to appraise a government’s performance. But sham numbers detract from those that inform. Shocking figures capture our attention, while colourful anecdotes eclipse the informative but mundane. If scepticism leads us to reject all statistics heading our way, the truthful numbers get filtered out as well, leaving us grasping in the dark. Even reliable statistics rarely convey the whole truth. Estimates have margins of error; averages don’t reflect diversity among individuals, and different measures can tell different stories.
Statistics that slice the cake in different ways provide the perfect opportunity for polarised debates. Cognitive scientists Hugo Mercier and Dan Sperber argue that our ability to reason didn’t evolve so we could discover truths. As social animals, it developed so we could justify our actions and decisions to others, thereby improving communication and cooperation. Accordingly, as the poet Andrew Lang once claimed, people use statistics like drunks use lamp-posts – for support rather than illumination.
Faced with arguments underpinned by numbers, we need to cultivate statistical alertness so we can spot the falsehoods but also read authentic statistics with shrewdness – learning from what they can tell us while acknowledging their limitations. Information is crucial in a democracy, but so is our ability to evaluate and interpret it. Placing statistics in mainstream education can help, but some simple checks can be applied.
Common sense should be our first line of defence. Highly exact or sensational numbers require particular scrutiny. Finland, the 2019 World Happiness Report told us, is the world’s happiest country, with a score of 7.769 out of 10 (Britain mustered 7.054). Is it reasonable to measure an entire nation’s happiness to three decimal places? A 2015 report stated that 82% of US adults sexted in the previous year. Really? Respondents can be less than honest. In a British study, 15% of people were willing to vent their views on the monetary control bill and 11% on the agricultural trade bill, neither of which existed.
But what about those numbers that penetrate our intuitive shield? If they are survey-based, we should ask how many people were questioned, precisely what they were asked and how they were selected. In 2015, the Daily Express reported that “the biggest poll in 40 years” showed 80% of Britons wanted to quit the EU. Two Tory MPs and a parliamentary candidate had trudged the streets of Northamptonshire delivering 100,000 questionnaires. Only 14,581 were returned. Not only was the sample unlikely to represent a cross-section of UK voters, there was also a danger of self-selection bias – the Disgusted of Tunbridge Wells (or, in this case, Wellingborough) phenomenon.
We should also be alert to definitions. The “number of Covid-19 tests” could refer to those issued, those used or those processed in labs. Definitions can be particularly troublesome in international comparisons. Not only are Finns supremely happy, they’ve never had an escape from an open prison. Prisoners making a run for freedom don’t escape, they are “absent without leave”.
Beware also percentages adorned with trailing zeros. A 400% year on year increase in offences linked to mephedrone use reported in Wales some years ago could have been an increase from one to five or 2,000 to 10,000 offences. And don’t be deceived by percentage-point changes masquerading as percentages. An increase in taxes from 35% to 40.4% is a 15.4% rise, not 5.4% as a US administration once claimed.
Finally, two figures don’t make a trend. “Gun homicides are up by two-thirds since 2013”, warned Canada’s public safety minister in 2018, justifying tightening restrictions on gun ownership. But 2013 was a convenient year for the comparison: it had the lowest rates of gun fatalities on record. Pick any other year – even back to the 1920s – and the apparent surge in wild west shootouts disappears.
H G Wells is often quoted as saying that statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write (though this paraphrases his actual words). In the so-called post-truth age, that day has undoubtedly arrived.