Lies, Damned Lies, and Statistics

Just because a number sounds plausible, it doesn't mean that it is accurate. So for young players, if you see it, don't repeat it. Never reference an article, broadcast, or news package without citing the source first. Find the raw data and if you can't find it, you shouldn't use it because it probably doesn't exist.
|

"There are three kinds of lies: lies, damned lies, and statistics". If it was American humourist Twain or Prime Minister Disraeli who first coined the phrase, really... Who cares? All we need to know is it's a bloody good one!

Because it's a fact. Never believe a statistic you read in print, hear on the radio or see on the television without questioning it first. Reason being, "86% of statistics are completely made up". Because while the numbers never lie, the statistics almost always do.

I am particularly weary of percentages used without hard data behind them. A journalist who uses these to validate their argument is either incompetent or lazy. Why? Because the first thing we learn straight out of the gates is to be cautious of statistical traps. There is no greater example than the sponsored survey.

Nothing is as 'conveniently catchy' as the press release from some sponsored survey trumpeting a big percentage headline. I don't know what outdated play book some PR companies are still using when they pitch these surveys, however they're pointless and a waste of their client's money because in this day and age well all know these surveys are rubbish.

My introduction to the sponsored survey was back in my talkback radio days when I was just a whelp working on a Sydney drive show. Some PR company offered up their client, a psychologist to talk about relationships and discuss the alarming new research that "42% of Australians Cheat on their Partners".

This troubling statistic would definitely be worthy of a debate if it were true and it had me a novice journalist fired up with my guns blazing. Luckily, my then boss, keen to teach me a lesson, instructed me to go back and quiz the spin doctor on their survey size before it went to air. I did this and reluctantly after several minutes probing they told me. To my shock horror, I discovered they'd only surveyed 1,000 people. That's right they were basing a whole nation's relationship habits on nothing more than 1,000 people and I nearly took the bait hook, line, and sinker!

In hindsight, I cannot believe how stupid I was not to question the percentage. How could 1,000 people give a clear indication on the practices of 23 Million? If this survey was accurate, and 42% of all Australians had in fact cheated on their partners, then by that count, it would mean we have over 9.6 Million philanderers running around down under!

That's nearly half the population! This purely isn't possible. Let's argue for a minute that people under 15 years old aren't getting 'any' and let's say people over 65 years old aren't cheating because they're too old to bother with such shenanigans. Right, based on raw data from the Australian Bureau of Statistics, that eliminates 4.4 Million and 3.1 Million Australians respectively.

So once we've cancelled out the elderly and the underage, there's only 15.5 Million people left in Australia aged 15-65. Of those, you would need at least 10 Million to actually be in committed relationships to make the 42% of all Australians supposedly cheating possible, but then the number would be closer to 100% of all people in relationships are cheating!

Don't get me wrong, it's a great headline, but either we're all terrible partners and going to hell or it is just plain untrue.

While that was targeted towards talkback and trashy tabloids, it seems these days no one is immune from dodgy numbers. Just last week, I even saw one the most venerated current affairs program in the land, Four Corners fall for an 'unquantifiable statistic'.

To give you a bit of background, Four Corners is an Australian institution. It's similar in many respects to the BBC's Panorama, and like Panorama, Four Corners also airs on our government sponsored broadcaster, the ABC.

Traditionally the bastion of investigative journalism in this country, it's a program of the highest standard and has always been beyond reproach. Maybe I am just nitpicking but I was very surprised when Four Corners dropped this 'clanger' last week.

When airing an investigation into the fishing industry, the following ludicrous statistic went unchallenged; they had a representative from the recreational fishing association claim "There are five million recreational fishers in Australia - about a quarter of the population."

Really? May I ask... How on earth would they know that? How convenient that it's such a high number coming from the recreational fishers. I for one never knew fishing was so popular. Who does the number crunching on that one anyway? I can tell you right now... I don't fish and I don't know anyone else who does either.

However, I did cast a line off a boat once in my teenage years. I believe it was around 1997 and I never did it again because of the sheer boredom. Does this mean I am one of the 5 Million recreational fishers? Is it people who "do" fish or "have" fished? There's a big difference.

While that granted, is a rather innocent little harmless fact open to interpretation the statistic can also be the main weapon of choice for scaremongering, abusing the facts and creating what we Journos call... 'A Beat Up'.

An example of a beat up is media outlets claiming that "On average 30,000 Australians go missing each year".

While that number might sound plausible in a population of 23 Million, break it down and see its faults; if this were true, it would mean 80 go missing per day or one person every 18 minutes which is completely impossible. If someone was going missing every 18 minutes never to return, I'm sure we'd know about it. Surely you'd notice it when your Christmas card list kept getting considerably smaller every year?

Sourcing the original Australian Institute of Criminology report its easy to see how the number is manipulated. While it did state on average 30,000 go missing every year, the next paragraph of that same AIC report, (a statistic never quoted by the newspapers) states that on average 27,000 are found that same week. Now that makes more sense. While far less dramatic, that figure of 30,000 now seems reasonable because it is not just people who never return, but includes teens who run away from home, shack up at a mate's and return three days later. It includes elderly family members who wander off for a day and it even includes other careless people in their late twenties who forget to call their mum to tell her they're going to Europe for a few weeks! (again, I'm really sorry about that!)

The point is that these numbers are far less dramatic when explained, so some journalists try to spice up their stories not by lying, but by 'omitting' facts. There's an excellent example of the role the media plays in statistics and scaremongering by Canadian Journalist, Dan Gardner's in his book Risk: The Science of Politics and Fear.

Gardner stumbled across a statistic that said "50,000 paedophiles were trawling the internet at any given time". Due to it being a 'conveniently round number' He immediately grew suspicious and questioned it.

Doing a bit of digging, Garner's search for the origin of this statistic led him through newspapers, the FBI, the office of the US Attorney General, NBC's Dateline, then right around back to the beginning.

What had happened? It began in the press as "there could be up to 50,000". The authorities quoted the press, then press quoted the authorities and it passed through various publications and government bodies until the number somehow became credible. The fiction became fact.

All we know, is no one knows the real number. 50,000 was a guess. Gardner traced this number back to find it used before.

During the 1980's it came up twice. First, where it was claimed that 50,000 children were kidnapped by strangers per year. Then secondly, at the end of the decade in was later suggested 50,000 was how many people are murdered in a year by satanic cults.

People think they can fool you with numbers. I can't speak for you, however I am more likely to respond and be won over by someone's thought provoking and meritorious debate. I can respect any argument, from a person with substance before I trust a person rattling off a bunch of statistical lies instead.

Just because a number sounds plausible, it doesn't mean that it is accurate. So for young players, if you see it, don't repeat it. Never reference an article, broadcast, or news package without citing the source first. Find the raw data and if you can't find it, you shouldn't use it because it probably doesn't exist.