POLLSTERS are a nit-picking lot. We have a great deal of experience trying to track the same opinions and attitudes over time and are familiar with the difficulties of doing so. Survey respondents are highly sensitive to different aspects of survey design, of which question wording is the most important. Changing an important survey question is, in general, a no-go – for very good reasons.
So, it’s right that we respond to today’s independence poll from Scotland in Union (SiU) with a great deal of scepticism. The standard independence voting intention question asked by pollsters is the Yes/No “Should Scotland be an independent country?” Instead, SiU semi-regularly ask the Remain/Leave “Should Scotland remain in the United Kingdom or leave the United Kingdom?”
The latest of these polls, released today, finds that 57% of Scots would vote to “remain” in the United Kingdom, once undecideds are removed. This is not comparable to the results of polls asking the standard question, so this cannot be compared to recent polls using the standard Yes/No question.
It is also highly questionable whether this question is reliably measuring propensity to vote for independence in the first place. We know that the standard Yes/No question does so, because following a long and hard-fought campaign, and a referendum with record turnout in 2014, Scots know what pollsters are asking with the Yes/No question. There’s no room for interpretation or misunderstanding.
READ MORE: 'Rigged' independence poll panned by pollsters and politicians
If SiU were reliably measuring the same underlying attitudes they should – in general – track the ups and downs of Yes/No polling. When Yes goes up significantly, reflecting real change in the population’s attitudes, we should see SiU’s “leave” vote go up as well, even if it does not increase by as much.
But we don’t see that similarity in tracking. Including undecideds, polls in October 2016 put the Yes vote on 42%, and SiU put the “leave” vote at 39%. In September 2020, when polls were consistently showing a Yes lead, and putting Yes as high as 51%, SiU still found a “leave” vote of 37%.
There are two key possible reasons why SiU’s question does not track attitudes to independence. Firstly, the question wording is loaded with emotive terms. Brexit, the Vote Leave campaign, and its proponents – who now sit in government at Westminster – are wildly unpopular in Scotland. In contrast, most survey respondents voted Remain in 2016 and many maintain an emotional attachment to the term. These emotional connotations may lead some to avoid saying they’d vote Leave, even if they would vote Yes.
Secondly, some respondents may simply conflate the question being asked with the Brexit question. This might sound condescending, but misinterpretation of questions is a regular occurrence in surveys and is why we should avoid ambiguous wording or – in this case – using wording closely associated with a different, very divisive issue.
While most people who would vote Yes would also tell SiU that they would vote “leave”, the effects of the question wording are clearly great enough to create a substantial gap between SiU’s findings and those of pollsters using the standard Yes/No wording.
SiU have been pushing for a Remain/Leave question – using the Electoral Commission’s recommendation for the 2016 EU referendum. They claim this question is fairer than the Yes/No question asked in 2014, forgetting that the Electoral Commission itself recommended the precise Yes/No question itself. So, they argue that because their question is fairer (it isn’t) that it better reflects Scots’ attitudes on independence (it doesn’t).
SiU’s independence polls are tinged by problematic wording of the questions. It’s not just that they don’t reflect the findings of other polling, but that they do not reliably track attitudes to independence in the first place. For that reason, nobody serious should take them too seriously.
Why are you making commenting on The National only available to subscribers?
We know there are thousands of National readers who want to debate, argue and go back and forth in the comments section of our stories. We’ve got the most informed readers in Scotland, asking each other the big questions about the future of our country.
Unfortunately, though, these important debates are being spoiled by a vocal minority of trolls who aren’t really interested in the issues, try to derail the conversations, register under fake names, and post vile abuse.
So that’s why we’ve decided to make the ability to comment only available to our paying subscribers. That way, all the trolls who post abuse on our website will have to pay if they want to join the debate – and risk a permanent ban from the account that they subscribe with.
The conversation will go back to what it should be about – people who care passionately about the issues, but disagree constructively on what we should do about them. Let’s get that debate started!
Callum Baird, Editor of The National
Comments: Our rules
We want our comments to be a lively and valuable part of our community - a place where readers can debate and engage with the most important local issues. The ability to comment on our stories is a privilege, not a right, however, and that privilege may be withdrawn if it is abused or misused.
Please report any comments that break our rules.
Read the rules hereLast Updated:
Report this comment Cancel