Most people following last year's Westminster elections were dumbfounded when the first results started to roll in. While pollsters had predicted a race too close to call, results suggested a clear victory for the Conservative Party. Indeed, of the last 10 pre-electoral polls, half of them predicted a tie between the conservatives and the Labour Party. Most of them suggested a deadlock at 34%. When all of the ballots were counted, it became evident that polls had not only made predictions outside of their margins of error, but had also failed to capture the true power asymmetry voters perceived between the two major parties.
The failure of polls raised serious doubts on the role of pollsters. Some argued that the polls failed to capture older voters who normally preferred conservative candidates; others argued that the polls failed to correct the fact that while under-30s voters generally lean left, they often failed to turn out on polling day. Most argued it was a combination of the two, and blamed the sampling methods in the polls - the failure to accurately represent the population through the sample. This sparked a series of pollster-funded studies, as well as independent reports, filled with corrective measures to improve the accuracy in forthcoming elections.
The elections held last Thursday, 5 of May - to elect the London Mayor, London Assembly, Northern Ireland and Welsh Assemblies as well as Scottish Parliament among others - were the first time polls were to be tested since their blunder in the 2015 Westminster elections. Of this electoral cycle, the London Mayoral election stands out as the best measure of whether the polls had improved on their diminished reputation. Like the Westminster elections, the London Mayoral elections are first-past-the-post, and because of the importance of the election and the familiarity of voters with the candidates, polls tend to measure voter preferences throughout the campaign.
The first polls fielded with aims to predict the outcome of the Mayoral election took place about a year before the event. In total, around 15 polls were fielded, with an average of two-per-month in 2016. The polls, mainly conducted by ComRes, Opinium, Survation, and YouGov, were all correct in pointing out that the Labour would win. Even the earlier polls, fielded before candidates actually registered, pointed out that irrelevant of the name of the competitors, the candidate fielded by the Labour party would win. The polls fielded latter on, after candidates were nominated, were equally consistent, pointing to a victory for Kahn.
When comparing the polls' predictions to the final result of each candidate, polls were somewhat less accurate. Overall, they were less accurate when predicting the result for Kahn than when they were predicting the result for Goldsmith. But, if only polls within two months of the election are considered, all polls came within three percent of accurately predicting the vote share of both Kahn and Goldsmith. Not surprisingly, polls were very accurate for all other candidates, who obtained an average of three percent of the vote. Naturally, it is easier to predict the result for candidates that have trouble growing outside the margin of error of polls.
When comparing the polls' prediction to the final result the election, polls are slightly less accurate. While the average of all of the polls is impressively accurate compared to the election result - most of them come to within 1 per cent of the final result - the individual predictions are less so. This is particularly true when considering the time of the election. The polls fielded in May and April were significantly more accurate than the polls fielded in March and February. The former group of polls were on average 3 per cent more accurate. This confirms an old rule of thumb, which holds that the closer to the election a poll is fielded, the more accurate it is.
If we compare these polls to the polls conducted in advance of the Westminster election, the most recent ones take the edge. The polls fielded for the 2016 electoral cycle consistently predicted the winner, they came within reasonable distance of predicting the share of votes of all of the candidates, they predicted the difference between the top two, and they were increasingly accurate during the campaign cycle. This is a restoration of the record of pollsters, who had their election cycle in recent times just a year ago. While we cannot be sure that pollsters effectively incorporated changes, it is safe to assume that they did indeed do something better.
Are the pollsters in the clear? Not yet. After the huge defeat in the Westminster elections they will have to prove that they will get better. The London Mayoral election shows a positive advance in a short period of time. However, they will only face the real test in the next UK parliamentary elections. Until then, pollsters have time to improve their methods. This essentially means advancing towards a more representative sample to get around the challenges that technological advancement have created. If polls cannot meet the challenge, they will be under more public scrutiny and pressure to shut down, and let exit polls take their place.