Despite being the most accurate of the pollsters at the 2015 election, the final ComRes poll was still not close enough to the actual result. So we have been conducting our own internal review of what happened.
As we stated in our “early thoughts” blog, we believe there are a number of reasons why the polls generally overstated Labour and understated the Conservatives. Our review will be ongoing but, with the release of our first post-election voting intention poll for the Daily Mail, we have applied a newly developed, innovative ComRes Voter Turnout Model.
Only 65% of registered voters cast their vote in May’s General Election. In order to achieve a more accurate reflection of an election outcome we need to be better at identifying the subset of the public who will actually vote.
ComRes, along with a number of other British pollsters, have traditionally asked respondents to rate their own likelihood to vote using a 1-10 scale. While this has proved a useful indicator of likely turnout, as ComRes colleague Andy White identified before the election, some survey respondents overstate their likelihood to vote.
In 2015 that overstatement was significantly more than in previous elections. In our final GE2010 poll, 70% said they were 10/10 certain to vote - but at GE2015 that figure was 78%, despite actual turnout increasing by just 1% point. Therefore, while we were already making adjustments for overclaim on turnout, our further investigations have led us to believe a stronger approach is needed to better reflect the voting population.
Our approach has been to look at the polls from an “external” perspective. That means looking at other data – actual results, census data, population projections, and so on – to ensure external anchor points for the design of our voting intention surveys.
By using this non-survey data we have now established an important principle: that the extent of this overstatement varies among different demographic groups.
Less affluent voters more likely to exaggerate likelihood to vote
In particular, our modelling of the election result, based on constituency-level turnout data as well as ward-level turnout data from local elections held at the same time, strongly suggests that less affluent voters are more likely to exaggerate their turnout likelihood.
Some have called this ‘Lazy Labour’, an unfortunately pejorative label, which also focuses too much on its effect on one party. In fact, when we looked at the 2010 results, we saw much the same level of overstatement in less affluent areas – especially those with high levels of ‘multiple deprivation’.
The modelling also identified a clear correlation between age, social grade and deprivation and turnout levels (particularly at the extremes of the scale), giving us a better idea of what the voting public actually looks like.
The ComRes Voter Turnout Model
Using all this information we have developed the ComRes Voter Turnout Model. This is by no means the only adjustment we will make for polling voting intention, but it is an important step towards using external data to better model actual voting.
By simulating turnout among different demographics we have developed a means of weighting the results to the expected turnout profile. The advantage of this model is that demographic turnout patterns are more consistent from election to election than the demographics of party support.
We have since gone back to our campaign polls to measure the impact of this model. If we were using this method for GE2015, and made no other adjustments to our methodology, our final poll would have had a 5 point Conservative lead over Labour. We have also applied this model to our first post-election voting intention poll. Were we simply using the same approach as before the election, our poll would give the Conservatives an 8 point lead. Using the new ComRes Voter Turnout Model the poll produces a 12 point Conservative lead.
As ever, we are not resting on this one solution and will continue our own internal review, making further (if probably smaller) adjustments where necessary. We also look forward very much to contributing to the BPC / MRS Review of the General Election.
As with all approaches, there is no one magic fix that is perfectly without flaws. Behaviours change as does the political landscape and with it so must our methodology. However, we do believe this to be a significant improvement on the accuracy of identifying the voting public and separating them out from the wider, general public. This model of course also has wider applications for all elections. We will therefore be applying this to other upcoming elections including the EU referendum as we continue to review, innovate and test approaches.