Polling – Why are they so wrong?

There's trouble in polling land. In election after election the polls keep getting it wrong.

Consider last summer's General Election. For almost two years the polls told us that the Labour Party was 20-30 points ahead of the Tories, and sometimes even more. When the votes were counted, they were just 10 points ahead. If any of us reported results to our boards that were 200% wrong, we would be an HR problem by mid-morning.

(Not to say we told you so, but for the record the REC team in our blog posts and presentations had been pointing this out for months on end and pretty accurately estimated the actual lead. If it had not been for the steep rise in Reform polling in the immediate run up to and during the General Election campaign, we would have had a horribly hung Parliament).

So why are the polls so regularly wrong? They didn't think Cameron would win a majority in 2015. They didn't think Brexit would triumph in 2016. They didn't predict the first Trump win. They didn't predict the Boris landslide in 2019. And they massively overestimated Labour's lead in the 2024 General Election. They didn’t foresee the Trump landslide last November. It's not a great record. So what's going on?

There are several reasons. Here are a few of them.

When our chairman was in short trousers (he's very old!) pollsters knocked on your front door with a clipboard. It was a truly random sample and, admittedly in simpler political times, the polls were pretty accurate. But it was expensive because you needed lots of people knocking on lots of doors.

By the time our chairman was a teenager with a suitably silly haircut, the pollsters had discovered telephone polling. And random digit dialling meant they could get a clean representative sample of voters fairly easily. The polls remained pretty accurate for a while. But it was still quite expensive because you needed lots of people manning telephones, and over time it became increasingly difficult to reach some groups, such as younger people who no longer had landlines and often seemed to have a phobia about answering the phone.

When the internet arrived, so online polls were born. Each polling company recruits a panel of people willing to be polled, for a small fee. And this was much cheaper because you just need to send out a whole lot of emails, your paid panellists do your work for you and all you have to do is analyse the results, produce some funky charts and send out a press release.

Problem 1: what kind of weirdo volunteers to sit on a pollster’s panel? Do you know any of these people? Well, we know what kind of weirdos, predominantly people with a centre Left persuasion apparently who dominate polling panels.

Thus Problem 2: this presents all sorts of challenges for the polling companies as they need politically balanced samples. So they resort to all manner of juju, magical mathematical trickery to end up with a balanced sample. It's called ‘weighting’. Weighting goes like this: ‘We have too many left-leaning rural mums in the southwest so let's dial that number down a bit and dial up our underweight white male Tory voters in eastern England’, or something like that. So the poor old pollsters are constantly having to twiddle a whole load of knobs on their models to try and desperately balance their samples.

Then we have Problem 3: we are living in crazy political times, in a new five party system, with swing voters all over the place, tactical voting galore, with new political parties popping up on a regular basis and with seemingly now regular ‘black swan events’ that move voters’ opinions substantially. It's a pollster’s hellhole. They keep having to invent new knobs to twiddle to try and get that perfectly balanced sample.

So dear friends our polls are not really very reliable at the moment. Our advice:

1. Always ignore a single poll as they are generally irrelevant.

2. Always look at the long term polling trend as this is likely to be more accurate.

3. But always listen to the sage-like advice of Lord Ashcroft: “Polls are a snapshot of current public opinion as at today and not a prediction of the future”.