On December 7th, the Economist published a bombshell poll. It found that 20% of Americans aged 18–29 agree with the statement, ‘The Holocaust is a myth’. That’s one in five. A further 30% neither agreed nor disagreed, which means that only 51% actually disagreed with the statement.
This shocking result was covered in numerous outlets, including the Hill, the Telegraph and the Daily Mail. It was also widely shared on social media. A tweet by Visegrád 24 garnered 11k likes. Another by Mike Bloomberg garnered 5.9k likes. And another by The Mossad: Satirical, Yet Awesome garnered 3.9k likes. There was even a spike in worldwide search interest for ‘holocaust myth’.
The only problem: it’s almost certainly wrong. When I first heard the result, I thought “that can’t be right”. But then I saw it was published in The Economist, who are usually pretty reliable when it comes to data and statistics. On this occasion, as it turns out, they were very unreliable.
We know the result is almost certainly wrong thanks to some work by Pew Research. The key issue is that it came from an ‘opt-in’ poll:
This is a type of poll in which the respondents have previously registered with a polling company to take part in regular online surveys. When a new survey is being carried out, respondents can opt-in until the required distribution of age, sex, education etc. has been achieved. Each one answers the questions anonymously and receives a small monetary incentive for taking part.
But because anyone registered with the polling company can take part, some respondents simply rush through the poll without answering questions sincerely. They may give ‘joke’ answers or just click randomly to save time.
Such ‘bogus respondents’, as they are sometimes known, usually don’t pose much of a problem. The only time they do is when the question concerns a belief or attitude that is rare (such as ‘the Holocaust is a myth’) and when you focus on a specific subgroup in which they are known to be overrepresented (like young people). Both these things were true of the Economist’s poll.
To find out whether the shocking ‘one in five’ result was due to bogus respondents, Pew Research attempted to replicate it in their January American Trends Panel – a survey for which respondents are recruited by mail using probability-based sampling. What did they find? Among Americans aged 18–29, only 3% agreed with the statement, ‘The Holocaust is a myth’.
Now, you might say that 3% is still high, but even in good-quality surveys there’s always a small percentage of respondents who don’t answer sincerely or who make errors when inputting their answers. In fact, the percentage agreement was exactly the same in every other age group, which suggests it’s nothing out of the ordinary.
Pew’s findings highlight the importance of not drawing strong conclusions from low-quality survey methods. If a result sounds wildly implausible, it probably is.
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.