With three polls in the last week from IPSOS-Mori, YouGov and Scottish Progressive Opinion, we anoraks have had plenty to chew over. They all employ quite different methodologies, so comparing them is a bit like trying to find points of similarity between broccoli, carrots and onions.
But they all indicate a significant shift in support towards the SNP. Without any doubt, the electoral sands are shifting in Salmond’s favour, though we should be wary of hailing the victors based on what is being reported, for there are some fine details that are worthy of consideration.
First, the figures are being massaged a little at the edges. This pre-occupation with dismissing don’t knows and uncertains to vote has a bearing on the findings. The IPSOS-Mori poll gives the most detail with which to examine this phenomenon. The headlines gave the SNP an 11 point lead on the constituency vote (45% to Labour’s 34%) and put them head by 10 points on the regional vote (42 to 32). But that finding is based on participants’ voting intention and which party they are inclined to support and how certain they are to vote. Which means rather than simply being a pure who will you vote for result, it has other things factored in as well.
One immediate consequence of doing this is to lower the sample size from the 1002 people polled to 681 respondents (or 667 if the weighted figure is used and we want to confuse matters further). Thus, the margin for error increases on a lower number and moreover, it is universally accepted that you need a sample of 1000 to make a survey fully representative.
If we include all participants, the sample size goes back up and the figures shift slightly. On the constituency vote it now looks like SNP on 43 and Labour on 34, a margin of 9 per cent, and on the regional vote, it is 40 to 32, a difference of 8 points. See below for how this might impact on seats in Holyrood.
The sample then drops dramatically for the question on which party people are inclined to support. Looking only at the findings for the constituency vote – I’m trying to simplify this, honest – we see that only 294 responded to the question and of that number, 88 were undecided or didn’t know and 73 refused to answer. This means that only 133 people gave a substantive response, yet this finding was applied to the whole sample to come up with the headline finding ie SNP on an 11 point lead.
So what happens if we take out all the extraneous stuff and simply ask all people whom they intend to vote for? We get a very different result. On the constituency vote, the SNP is on 29%, Labour on 20%, Tories on 7%, Lib Dems om 6% and the don’t knows are on 23%. For the list, it’s 28% SNP, 21% Labour, the Tories and Lib Dems on 7%, Greens on 4% and the don’t knows on 18%.
It is still a commanding lead for the SNP but suddenly, the large numbers of don’t knows – as also found by Scottish Progressive Opinion -Â become more relevant.
The adjustments for certainty to vote are much more subtle in YouGov’s poll findings but it is not clear how many people fall out of the sample because that detail is not given.  However, the vagaries of the electoral system mean that even a percentage point of difference – Labour is on 32% when the findings are adjusted compared to 33% when they are not – results in Labour gaining another couple of seats when those findings are entered into ScotlandVotes predictor.
It is also worth noting that YouGov’s sample size has increased considerably. In March’s poll, it was 1025; last week’s Scotland on Sunday poll was based on a sample of 1135 adults; and this week’s sample was 1332. It should make for a more accurate picture but there is still considerable weighting being applied, one presumes on the basis of Westminster voting record rather than previous Scottish Parliament vote share.
The general point holds true: every time the raw data is poked around with, no matter how stringently rules are adhered to, there is a risk of contamination and affecting the findings.
And every time don’t knows are airbrushed out of the equation, the findings are being skewed somewhat. Some of these will be genuinely unsure voters who will make up their minds at the last minute, some as late as when they reach the polling booth; others will actually be won’t says ie they know how they are going to vote but they won’t share it; others can now be considered as won’t votes.  At this late stage of the campaign, if people have not made their minds up, often they simply will not bother to vote at all.
So while the large numbers of undecideds might still give the Labour party a glimmer of hope, their potential for causing a swing back is diminishing day by day. Indeed, some will simply make for the winners’ bandwagon which is more good news for the SNP.
Finally, a percentage poll lead does not translate into a gain by that margin. A ten point poll lead translates into a much smaller swing which means most Labour MSPs will escape, albeit with their majorities scythed. The SNP’s potential gains are just as likely to come from the Tories and the Liberal Democrats but in all cases, local factors and incumbency come into play. On the face of it, Ayr and Galloway and West Dumfries should fall but both have longstanding, respected Conservative MSPs. Moreover, both Tavish Scott and Iain Gray are unlikely to lose their seats and Kevin Stewart in Aberdeen Central, as depute leader of a council forced to make horrendous cuts to balance the books, might find it harder to shift an incumbent MSP than all the polls suggest.
It all adds up to a great big headache and a couple of truisms:Â all to play for and the only poll that counts is the real one on 5 May.
#1 by John Ruddy on April 25, 2011 - 12:58 pm
A good point about Aberdeen Central. Lewis has been the MSP there for 12 years – he is very well known, and respected in the area for the work he has put in.
Kevin Stewart is depute in an unpopular local council. Every tiny decision by that council will be used against him, as happened in Glenrothes when increased alarm charges for elederly folk was played out against the SNP candidate (also head of the council).
The number of don’t knows not only give Labour some hope (I think a lot of the recent SNP gains are “soft” SNP voters – they may also change before polling day, and not back to where they came from), but they should also be a warning to all politicians. Why do we have so many people not engaged in the political process?
#2 by Doug Daniel on April 25, 2011 - 9:17 pm
Interesting that you say the number of “don’t knows” should give Labour some hope – in my eyes, “don’t know” is not too far away from “don’t really care”, in which case such people are more likely to vote for whoever the current government is, since whoever is in charge obviously hasn’t done anything bad enough to make them care about the election, and thus boot them out of office.
What makes you think that “don’t knows” are more likely to convert into Labour votes than SNP votes?
#3 by John Ruddy on April 25, 2011 - 10:22 pm
I said some hope. However, I also pointed out that lack of engagement in the political process – ie people who wont vote. I think turnout will struggle to get over 50% – yet Westminster gets 70% (though both ought to get more).
#4 by The Burd on April 26, 2011 - 12:16 am
Not about that at all, just that the fact there are so many don’t knows means there are still votes to win. It means they have a slight hope still of closing the gap. And I did say slight. I can remember being at this stage in campaigns and clutching at all available straws….
#5 by Malc on April 25, 2011 - 1:12 pm
This really is a cracking analysis Kate. Thanks for sharing it on Better Nation!
Its stuff the papers either haven’t picked up on or have picked up and have just ignored. It still suggests the same outcome – but we are certainly looking at a much tighter race than the topline figures suggest. But I suspect the parties themselves know that as well.
The local factors are notoriously hard to factor into any analysis – especially with regards opinion polling. You only need to look at 2007 to see that: large-ish swing to the SNP, which saw them pick up seats like Gordon, Stirling and Edinburgh East & Musselburgh but left seats like Cumbernauld & Kilsyth as Labour holds. How do you put a figure on incumbency and/or the strength of particular candidates/parties locally? No idea – I think some of the predictions I’ll be making will be pure guesswork when it comes to some of them.
Fab stuff – and I hope you make your stay here permanent. 🙂
#6 by Ezio on April 25, 2011 - 2:05 pm
I’ll never understand why parties think putting up people who run local councils is a good idea. Are these people ever popular?
#7 by Doug Daniel on April 25, 2011 - 9:26 pm
There’s an idea for a good bit of analysis in a future blog post, if anyone is up for it (probably post-election): how many MSPs are ex-councillors, specifically high-ranking ex-councillors? It would be interesting to see if the council-to-Holyrood route is a road well trodden, or if becoming a councillor is a sure-fire way to ensure your political career goes no further than the Town Hall.
I’m guessing the main reason they do it is because a well-known politician with some bad press is better than someone completely anonymous. Or the councillor in question is just deemed to have performed extremely well and outgrown the confines of the council. Who knows?
#8 by Daniel J on April 25, 2011 - 10:04 pm
I don’t know the exact figures but I know Holyrood has a pretty high rate of people from “politics facilitating careers” and I think there are an awfully high percentage of ex-councillors in there (and former MSP/MP staff of course).
#9 by Doug Daniel on April 25, 2011 - 10:11 pm
Westminster is also full of ex-political advisors etc – it would be interesting to know if this is a particularly British phenomenon, or if it is the same across Europe. I imagine in a not too distant future, MPs and MSPs who didn’t “rise through the ranks” so to speak will be a very small minority indeed.
#10 by The Burd on April 26, 2011 - 12:07 am
And that Doug would be a very sad day.
#11 by The Burd on April 26, 2011 - 12:08 am
The suggestion of some analysis on this is a good one and not just of ex councillors but of everyone from the inside of politics
#12 by douglas clark on April 25, 2011 - 2:42 pm
I think this is an important article. But I’d like to ask Kate what I hope is a straightforward question.
If Polls are conducted on a genuinely random basis then the ‘don’t knows’ are almost certainly ‘won’t votes’, no? Indeed comparing the results shown here to East Lothian, it is those that are declaring a preference that are overstated.
At the last election in that constituency only 56.2% of the electorate turned up to vote.
I think we need the information on ‘won’t votes’ provided separately before we can draw any definite conclusions.
#13 by Gavin Hamilton on April 25, 2011 - 4:04 pm
I tnink this is excellent in terms of truly understanding the polls and you begin to see why one pollster might always seem to underestimate a given party and overestimate another – albeit by a percent or two.
I am expecting incumbanc will mean a few survivals when the swing suggests otherwise. But under our system expect the List to even things out to close to what the polls are suggesting as the final answer.
The other factor I expect to see is a tightening of the race in the last few days. This seems to happen in many elections the world over. I expect that here and i epect Labour to go hard at SNP in the last few days.
I expect SNP to win and Labour to be a bit disappointed but I expect things to tighten!
#14 by Bill Pickford on April 25, 2011 - 4:53 pm
Gavin says:
“I expect SNP to win and Labour to be a bit disappointed”
A bit disappointed? They’ll be bloody incandescent!
#15 by mav on April 25, 2011 - 7:00 pm
galloway is interesting too. Alex Fergusson was elected last time round with an increased majority which was very much a personal vote. He was the pressed into being Presiding Officer despite his concerns about how his constituents would see it. If he now lost his seat, I suspect it would be a while before a constituency MSP was willing to serve as Presiding Officer.
#16 by Kate on April 25, 2011 - 7:39 pm
The number of don’t knows is very interesting and interesting that they’ve been removed from the equation by the media…. Douglas, won’t votes are also listed, as are refused to answers. I think about 10% of the don’t knows by this stage are probably won’t votes. EAch day they stay don’t know they increasingly become won’t votes. But at the moment they cannot simply be airbrushed out of existence, esp as they are higher in some key voter and swing groups like women and C2DEs. Whoever can persuade these folk over the door, can win.
Malc, how kind…. shucks!
#17 by mav on April 25, 2011 - 9:31 pm
Fron what I gather, each polling company has a different way of dealing with the don’t knows, and the maybe won’t votes. Some simply ignore, while others (yougov for example) ask you how certain you are to vote, 1-10, and weight you accordingly, by probability. Each number gets a percentage, based on the pollsters experience. So if they reckon that 50% of those who assess them as 7 won’t vote, then they discount you down.
Another factor is who they then weight other factors. They try to make sure they get the right demographic balance, so they ask stuff like who you voted for last time, or what paper you read. It kind of makes sense – if you have a sample where only 5% are saying they voted Conservative a year ago, and the Conservatives scored 23%(?) in Scotland, then you’re probably undersampling the Conservatives. (They also tend to assume that the don’t knows will end up going the way they went last time). From what I gather, online polling has a problem finding enough low income voters, so YouGov tend to weight up those people. Face to face polling meanwhile tends to have the opposite problem, apparently. Remember 12 months ago (and the Clegg bounce with two weeks to go)? The only time the polls reflected the final answer was the exit poll. I understand the polling companies have been navel gazing since and decided that was the only poll with a really standard sample, and that it was their samples combined with their weightings that were wrong, and had they got the weightings right, they’d have been more accurate. They are now using new weightings which may or may not be correct for Scotland.
#18 by The Burd on April 26, 2011 - 12:15 am
Thanks for this – the weighting on the voting intentions is quite interesting and still assumes a built in Labour majority.
It will be absolutely fascinating to see if the new weightings are correct or not!
#19 by Brian on April 25, 2011 - 9:41 pm
This article strikes me as an attempt to portray the polls as incomplete, but fails to take into account the decades of experience of the polling firms.
The questions are raised to reduce confidence in the results, and various innuendos are used to attempt to justify that conclusion.
For example, much is written about those who answered as dont’ knows. One hypothesis is that they are truly undecided, another that they are not voting and each party partisan claims the lion share for their party. One hypothesis rarely put forward is that many answer don’t know as a polite way of stating “none of your business”.
Polling firms are aware of this and their weighting and conclusion take this into account.
Barring any legitimate criticism of the actual numbers, and not just innuendo that tries to create doubt, we would all be better off accepting the polls for what they are, a snapshot taken at a point in time.
#20 by The Burd on April 26, 2011 - 12:14 am
Not at all. The media clients have been reporting very topline figures and not giving the detail which make the results that give them a headline much less definite. This blog was an attempt to point some of that out – that sample sizes and how findings are arrived at all have an effect.
It’s not about reducing confidence in the results but to address the rather sensational headlines being reported. And your last point about don’t knows is made.
And no, we should never accept anything that is presented to us as fact without questioning it actually.
#21 by douglas clark on April 25, 2011 - 9:42 pm
Kate,
Thanks for the reply.
My polling point is this I suppose.
If you sample 1000 people then – according to this:
http://tinyurl.com/6e8upvo
if folk are being ‘honest’ then say 40% of them didn’t vote at the last election. Shouldn’t the poll also accurately reflect, approximately that figure?v Unless I am reading the YouGov Poll completely wrongly the won’t vote is running at 1%! On both the constituency and regional lists.
Frankly, that doesn’t make any sense to me whatsoever, so I must be wrong somewhere, but if I am, I don’t know how. An unknown, unknown in Rumsfeld speak.
#22 by The Burd on April 26, 2011 - 12:11 am
People fib and forget! Elections – to most people! – all merge into each other and folk can scarcely remember what they did the last time. So folk guess and others don’;t want it known they don’t or didn’t vote. Likewise the 67% absolutely certain to vote which someone else already commented on. All of this, one assumes, is factored into the weighting.
#23 by douglas clark on April 25, 2011 - 9:48 pm
mav,
Thanks for your clarification. It just surprises me that, if a polling company asked you whether you voted or not at the last equivalent election and you answered ‘no’, that that in itself ought to have by far the largest weighting.
#24 by The Burd on April 26, 2011 - 12:09 am
I dont’t think that question is being asked in these polls which is why all the weighting – and thanks to mav for the full low down on how it works. We were getting way beyond my competence/knowledge in this area!!
#25 by douglas clark on April 26, 2011 - 12:28 am
Kate,
Y’know that silly poem:
“Big fleas have little fleas,
Upon their backs to bite ’em,
And little fleas have lesser fleas,
and so, ad infinitum.”
Well, I’m way down at the ‘so on as infinitum’ level!
I didn’t even get the joke headline:
‘I d’Hondt like Mondays’
See, you?
You know about stuff
🙂
_____________________
But it still seems to me I have a point worth discussing.
If circa 40% of folk don’t vote and are surveyed, why is it the case that that translates to circa 1% declaring that they won’t vote?
I don’t see how you can get a realistic answer if that anomaly isn’t fully explained.
I’d be quite happy to learn.
Anyone?
#26 by Dave Summers on April 26, 2011 - 11:32 am
good article Jeff. This time 4 years ago the SNP were 12% ahead with progressive poll company – and ended up 0.7% ahead – and lots of spolied ballot papers. Lat year Mori had the SNP 2% behind Labour for Westminster and they ended up 22% behind. Scottish opinion polling? not worth the paper its written on!
#27 by Jeff on April 26, 2011 - 11:45 am
Thanks Dave. It was actually Kate that wrote it though….!