We’re publishing an original essay today by regular contributor Mike Hearn about the problem of ‘pro-social bias’ in online opinion polls. What this means is that the huge public support for the Covid restrictions revealed in opinion polls for YouGov and others is likely to be overstated. Here is an extract:
Online panel polling solves the problem of low phone response rates but introduces a new problem: the sort of people who answer surveys aren’t normal. People who answer an endless stream of surveys for tiny pocket-money sized rewards are especially not normal, and thus aren’t representative of the general public. All online panel surveys face this problem and thus pollsters compete on how well they adjust the resulting answers to match what the ‘real’ public would say. One reason elections and referendums are useful for polling agencies is they provide a form of ground truth against which their models can be calibrated. Those calibrations are then used to correct other types of survey response too.
A major source of problems is what’s known as ‘volunteering bias’, and the closely related ‘pro-social bias’. Not surprisingly, the sort of people who volunteer to answer polls are much more likely to say they volunteer for other things too than the average member of the general population. This effect is especially pronounced for anything that might be described as a ‘civic duty’. While these are classically considered positive traits, it’s easy to see how an unusually strong belief in civic duty and the value of community volunteering could lead to a strong dislike for people who do not volunteer to do their ‘civic duty’, e.g. by refusing to get vaccinated, disagreeing with community-oriented narratives, and so on.
In 2009 Abraham et al showed that Gallup poll questions about whether you volunteer in your local community had implausibly risen from 26% in 1977 to a whopping 46% in 1991. This rate varied drastically from the rates reported by the U.S. census agency: in 2002 the census reported that 28% of American adults volunteered.
Worth reading in full.
