AFTER an hour or two savouring the SNP’s spectacular showing in the exit poll, supporters of independence wouldn’t have known whether to laugh or cry when Sir John Curtice appeared on screen to explain that the poll was likely to prove highly accurate – apart from the small matter of the Scottish part of the prediction, about which he had no great confidence at all.
The problem was that there had only been around a dozen sampling points in Scotland, which wasn’t sufficient to ensure the same level of reliability achieved elsewhere.
It was, he said, very difficult in the context of a Britain-wide exit poll to cover Scotland properly.
It’s hard to think of a better illustration of how Scotland was treated as an afterthought in this campaign, because it would of course have been perfectly possible to cover Scotland properly if the broadcasters who commissioned the poll had recognised that extra investment was required to take account of the fact that this country has its own political system and thus poses particular polling challenges.
READ MORE: John Curtice dispels myth that Labour voters oppose independence
The obvious solution to a lack of sampling points would have been to use more sampling points. One of the main things the exit poll got wrong in Scotland was that it predicted the Liberal Democrats would be wiped out completely, whereas in fact they took four seats.
That begs the obvious question of whether any of the sampling points were in the five LibDem-SNP battleground seats, which were bound to behave somewhat differently from the vast majority of constituencies in which the LibDems have relatively trivial levels of support.
And even if polling was indeed conducted somewhere in those five seats, it almost certainly would have happened in only one of them, leaving open the possibility of a major error if that one seat behaved in an atypical way.
The result in East Dunbartonshire was certainly very different from the LibDem wins in Edinburgh West and North-East Fife.
Earlier in the campaign, the lack of media interest in what was happening north of the border meant there were no full-scale Scottish polls for several weeks. That would have left us totally in the dark about the state of play if it hadn’t been for the fact that YouGov appear to correctly structure and weight their Scottish subsamples for GB-wide polls, meaning that an average of subsamples over a period of time can give a reasonable clue as to what a full poll would reveal.
READ MORE: Nicola Sturgeon doubles down on indyref2020 commitment
And thankfully they provided reassurance that the SNP vote was holding up impressively, although they also correctly suggested that the SNP’s lead was being cut back due to a Tory surge at the Brexit Party’s expense.
It was perhaps unfortunate that the long wait for a full-scale poll was ended by Panelbase, which in recent times has consistently reported less favourable numbers for the SNP than other firms. That led to what in retrospect were unwarranted worries about the SNP failing to make really telling progress against the Tories.
Until the results came in there was no way of knowing for sure which firm’s methodology was producing the most accurate results, but it’s now clear that Panelbase lost the battle of the pollsters on this occasion. Every firm ended up underestimating the SNP, at least slightly, and every firm underestimated the gap between the SNP and the Conservatives.
But Panelbase were much further out than the others – their final poll had the SNP on just 39% of the vote, a mere 10 percentage points ahead of the Tories on 29%. The actual result saw the SNP take 45% of the vote with a 20-point advantage over the Tories.
The closest that any poll came to predicting that outcome was an Ipsos MORI survey which put the SNP on 44% and the Tories on 26%. Ironically those figures were probably disbelieved more than any others, because after the bruising experience of 2017 there was a tendency to assume that the true position must be worse than the least favourable poll, and that the most favourable poll was likely to be miles out. But in fact there was always good cause for wondering whether the SNP might be underestimated this time, rather than overestimated. The most important reason for pollsters getting it wrong in 2017 was that a disproportionate number of people who voted SNP two years earlier had simply stayed at home.
Most firms subsequently introduced weighting by recalled 2017 vote, and would have expected it to correct any inaccuracies in relation to the SNP. But there was always a danger of that proving to be an over-correction if the 2017 abstainers turned out this time, and that’s exactly what seems to have happened.
READ MORE: Sturgeon warns Corbyn against stalling indyref2 until 2021
Even the much-vaunted YouGov constituency projection model had a patchy record during the campaign. It correctly spotted that the SNP were starting to make inroads in Tory-held seats, but at the same time suggested that the SNP’s hopes of gaining seats in the central belt were being hit by a localised Labour comeback. That simply didn’t materialise.
The model also gave succour to the confident claims from the Tories that they were on course for an astonishing gain from the SNP in Lanark and Hamilton East, but that proved to be well wide of the mark.
Going forward, it’s important to remember that polling methodology is continually being changed and refined, and that when we look for errors in future campaigns we shouldn’t necessarily expect to find them in the same places.
It’s possible, for example, that Panelbase will learn from their mistakes and become one of the more accurate firms next time around.
It’s also conceivable that the high SNP turnout may lead to another over-correction, and that the SNP may once again begin to be overestimated by the polls. But would that be such a terrible thing?
In any stand-off between the Scottish and UK governments over a future independence referendum, an enhanced polling lead for the SNP could actually prove to be a rather useful piece of ammunition.
Why are you making commenting on The National only available to subscribers?
We know there are thousands of National readers who want to debate, argue and go back and forth in the comments section of our stories. We’ve got the most informed readers in Scotland, asking each other the big questions about the future of our country.
Unfortunately, though, these important debates are being spoiled by a vocal minority of trolls who aren’t really interested in the issues, try to derail the conversations, register under fake names, and post vile abuse.
So that’s why we’ve decided to make the ability to comment only available to our paying subscribers. That way, all the trolls who post abuse on our website will have to pay if they want to join the debate – and risk a permanent ban from the account that they subscribe with.
The conversation will go back to what it should be about – people who care passionately about the issues, but disagree constructively on what we should do about them. Let’s get that debate started!
Callum Baird, Editor of The National
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereLast Updated:
Report this comment Cancel