Time for a Cool Head and an Open Mind?

Posted on 20 May 2016 by John Curtice

It has been a week of considerable polling excitement in the EU referendum campaign. First, on Tuesday the Daily Telegraph reported a phone poll from ORB that, once Don’t Knows were left aside, put Remain on 58%, up four points on the company’s last phone poll in the third week of April. Then at Wednesday lunchtime the Evening Standard carried a phone poll from Ipsos MORI that, on the same basis, gave Remain 60%, also up four points on its last poll conducted a month ago. Indeed, this was the highest figure for Remain in any poll since February.

Coming as they did the week after interventions from Mark Carney, the Governor of the Bank of England, and from Christine Lagarde, Managing Director of the International Monetary Fund, advising of the economic risks they believe are associated with a vote to Leave, these poll findings appeared to many commentators as evidence that voters were beginning to pay heed to the warnings about the economic consequences of voting to Leave. Moreover, it also seemed to many that these polls belied the message of the internet polls that the race remains as tight as ever. Certainly that appeared to be the view of the financial markets, with the pound staging a notable rally after the publication of the Ipsos MORI poll.

But one of the golden rules of the reporting of polls is that those with exceptional or dramatic findings secure widespread publicity, while those that are more mundane attract less notice. In fact, no less than seven polls have been published this week, including, not least, another phone poll from ComRes for ITV News and the Daily Mail that was released last night. Between them they make for a much more prosaic story in which it is far from clear that there has been any movement in the polls at all.

Below is a table of all of the polls that have been published this week, showing the percentage for Remain and Leave once the Don’t Knows have been removed. The penultimate column shows the increase or decrease in support for Remain since that poll was last conducted. However, in some cases that was no more than a week ago, while in others it was the middle of last month. To provide a consistent basis for comparison we also show for all of this week’s polls how their most recent figure compares with that in the equivalent poll a month ago.

Table. Polls published in week commencing 16 May

20 May table

There are a few details to bear in mind. The first concerns timing. The oldest poll, from TNS, was largely completed before Mark Carney’s warning on 12 May and that of Christine Lagarde on 13 May. It thus could not have picked up any impact their comments may have had.  ORB’s interviewing began before both pronouncements but was not completed until their headlines had hit the air waves. It may thus be at risk of underestimating any impact they may have had. All of the remaining interviewing was conducted after Mr Carney’s intervention and in nearly all cases, Ms Largarde’s too, though of course we may wonder whether those who were completing ICM’s surveys over the weekend had necessarily processed the information that their pronouncements contained.

The second concerns methodology. In the poll that ComRes released last night, the company changed the basis on which they quoted their headline figure. Hitherto this has been the reported vote of all respondents, though this has always been published alongside ComRes’ estimate of what the outcome would be after taking into expected demographic differences in turnout. (Those expected differences are not just based on voters’ reported likelihood of voting, as in the practice of many other polling companies, but on an analysis of turnout in previous ballots. In contrast to weighting schemes based on reported likelihood of voting, ComRes’ scheme anticipates that voters who support Remain will be more likely to vote than those who back Leave, adding two points to the Remain tally.) The figures for the change in the Remain vote for their latest poll in our table are thus the changes on the company’s turnout-weighted figure in their last poll rather than the headline figures that were published at the time.

Meanwhile, YouGov have also implemented a change in their weighting methodology, drawing a distinction between those who identify with an ‘established’ party such as the Conservatives or Labour, and those who identify with a ‘challenger’ party such as UKIP. The change has been made in reaction to the company’s estimate of UKIP support in the elections on May 5th. Its effect was to add one point to the Remain tally – and thus effectively more or less accounts for the small increase in Remain support registered by that poll.

These details aside, one things is clear. The substantial movements identified by ORB and Ipsos MORI have not been identified by any of the other polls this week, including two (from ComRes and YouGov) that were conducted well after the dust from Mr Carney and Ms Largarde had had some time to settle. Moreover, it is not simply the case that those polls conducted by phone consistently have identified a substantial swing that internet polls have failed to identify. Neither ICM’s phone poll nor that from ComRes have replicated that finding – both say that there has not been any significant change. But of course what is true is that, after a dearth of phone polls, the psychology of the week has been affected by the appearance of four phone polls that all certainly affirm the message of previous phone polling that the race does not look anything like as close as suggested by those polls done over the internet (and it is their appearance that has moved our Poll of Polls to Remain 55%, Leave 45%).

Meanwhile, there are two other findings in this week’s polls that are inconsistent with the thesis that the two interventions might have made a difference. First, if they had done so, we might have anticipated that the Remain side would have made particular progress in persuading voters of the merits of their economic case. Yet this is not what ORB found. True, they do report a four point increase on their previous poll in the proportion who think that remaining is more likely ‘to create a stronger economy’. But they also report an equally big switch in favour of Remain on the issue of immigration too.

Second, one of the striking findings in ComRes’ recent poll is that the proportion who say that the economy is one of the three key issues that will be important to them in deciding how to vote has increased from 38%, when they last asked the question in February, to 55% now. That would seem to signal a success by the Remain side in getting voters to focus on an issue on which they are inclined to believe that Remain has the stronger argument. Yet this has not been translated into a marked increase in support for Remain. (If reported on the same basis as last night’s poll, Remain stood at 58% in February). In part the increase in the proportion saying that the economy is important may simply reflect a greater polarisation between Remain and Leave supporters in their perception of the importance of the economy. While it was already the case in February that Remain voters (47%) were more likely than Leave supporters (29%) to say that the economy was an important issue, that gap has now widened to Remain voters, 68%, Leave supporters, 39%.

So, given that it is not clear that the interventions by Mr Carney and Ms Lagarde have made much, if any difference, what does the pattern of change look like if we take into account all of the polls published this week irrespective of when they were conducted – and compare them consistently with that they were saying a month ago. The answer is to be found in the bottom right hand corner of our table. On average the change in the level of support for Remain is zero!

Of course, it may be that concerns about the economic consequences of leaving the EU will be a slow burning issue for Remain that eventually moves public opinion in its direction. Perhaps the substantial swing to Leave reported by TNS will prove to a bit of a ‘rogue’. But for now what we do have to conclude is that collectively the latest round of polling does not provide clear evidence that a swing to Remain is in motion. The lesson of this week is that, as polling day begins to approach, it will become increasingly important to read the polls with a cool head and an open mind.

John Curtice

By John Curtice

John Curtice is Senior Research Fellow at NatCen, Professor of Politics at Strathclyde University, and Chief Commentator on the What UK Thinks: EU website.

There are 48 comments

Your email address will not be published. Required fields are marked *