Can we trust opinion polls?

Political opinion polls have taken a bit of a battering in the past few years. There was wide agreement on the eve of the last UK general election that the outcome would be a hung parliament. Few pollsters saw Donald Trump winning the 2015 presidential election in the US. And almost everyone agreed that Brexit would not happen. So are opinion polls worth the paper they’re written on any more?

Polling made its entrance onto the political stage in the United States presidential election of 1936, at a time when various prominent American newspapers were confidently predicting victories for Republican Alf Landon on the basis of polls of their (rich, unrepresentative) leaderships. George Gallop realised that he could achieve much more accurate predictions reasonably cheaply by taking a random sample of the population, and by doing this successfully forecast a landslide victory for Franklin D Roosevelt.

The key words in this statement are “random sample”, and this is where modern day polling is running into trouble. When Gallop began building his market research empire, gauging public opinion was a complicated business. It involved sending trained interviewers out to randomly selected addresses to interview a specific named person. If they couldn’t get hold of them, they were asked to go back again and again until they found them. What pollsters call “response rates” – the proportion of people agreeing to be interviewed – were very high. So was the cost. You had to train your interviewers, send them out, and tabulate the results, which in the BC years (before computers) was done by hand using punched paper index cards.

However, overwhelmingly, results were good, politicians came to rely heavily on poll predictions, and newspapers got into the habit of using them in order to report politics as entertainment about who was winning.

Polling today

These days technology and changes in the ways political opinion polling is done allow market researchers to get answers much more quickly and cheaply. Polling can also be done by post, online, or by phone. Rather than genuinely random samples, it’s usually cheaper for market researchers to use what are known in the trade as “quota samples”. Interviewers talk to certain numbers of people in different demographic categories (by gender, income, social class, ethnic group and so on).

However, they face several increasingly difficult challenges. Some kinds of people are just harder to reach than others, especially people who work full time – a group who are still a bit more likely to vote for conservative parties. We are now asked our opinions about so much so pointlessly that response rates for polls are desperately low at around 25-30%. We all suffer from poll fatigue.

Respondents are also self-selecting. People who are interested in politics are more likely to be willing to share their views with a stranger, and also are more likely to be left wing. All of these factors mean that the samples used by the pollsters to make their predictions simply aren’t as good as they used to be, and they all tend to err in the same direction.

This doesn’t mean that polls are now redundant. Well-constructed surveys which are properly carried out still get representative results. For example, the sample used by the British Social Attitudes survey, carried out via face-to-face interviews and requiring revisits where the randomly selected individual was unavailable for interview, correctly forecasted around a six point lead for the Conservatives in the 2015 general election.

However, these high quality polls are expensive, and take a long time. Given that the mass media mostly wants poll numbers rapidly, and for entertainment, it hardly seems likely that they will want to make the extra investment. Parties’ own internal polls do take the time and trouble and do get accurate results, ones which will no doubt have been part of the prime minister’s decision to go to the country. Current published polls show the Conservative Party has a 20 point lead over Labour, if not more. Is the true situation in the country likely to be anything other than a large Tory lead? Absolutely not: even cheap polls are not that inaccurate. As it stands, you’d be most unwise to take the 12:1 odds currently offered by some bookmakers on Labour being the largest party on June 8.