The changing moods of Brexit, or why we shouldn't necessarily believe YouGov

A couple of recent polls have given renewed vigour to those campaigning for a second referendum. In one case it was suggested that 53% of the British public were now in favour of remaining (1); whilst another found that 112 leave voting constituencies had changed their minds and would now support remain (2).

This has led to a flurry of articles in newspapers, and activity on social media suggesting that the mood has changed and that the logical conclusion of this change was that the demand for a “People’s Vote” should be conceded by the Government. In addition, campaigners for a second referendum have been targeting the alleged ambiguity of the Labour Party and suggest that it would be in their electoral interest to back a ‘People’s Vote’(3).

There are a number of issues which those supporting the People’s Vote campaign need to address. First of these is that in order to have a second referendum it would be necessary to get a vote through Parliament. It is not clear why the Government, which has not wavered in its support for Brexit, would support such a vote. Indeed, they seem convinced that the public are now fully supportive of Brexit (nobody they have spoken to in their local Conservative clubs has told them otherwise presumably!) (4). 

Not much enthusiasm for a second referendum here

Second, at no time did anybody suggest that the referendum was the first of many. Indeed, the literature put out by the Government at the time made it quite clear that the referendum result would be honoured. Therefore, the case for a second referendum on the basis of a shift in public attitude (if indeed there has been a shift) has no legal basis, even if it is desirable. 

Third, many remain supporters have claimed that people did not really understand what they were voting for in the first referendum. Given the complexities of unravelling the relationship with the EU this is undoubtedly true. But this then begs the question, how would we be any better informed in another referendum?

Finally, the shift demanded of Labour is contrary to Labour’s agreed conference position and seems designed as much to embarrass the Labour leadership, given those behind it (Peter Mandelson (5), Alistair Campbell (6), Chuka Ummana (7)), as to achieve a second referendum. This is not to say that Brexit is a good idea, as I have stated previously (8), whilst I don't want to be a bad loser, I'm still far from convinced that leaving the EU is anything but a disaster waiting to happen. The Government's appalling inability to negotiate a deal is evidence that so-called 'Project Fear' is fast becoming 'Project Reality'.

Perhaps more importantly the view that the public mood has changed is not as clear cut as those wanting a second referendum (based on the fact that they think they would win, which surely is a good reason for the other side to frustrate that wish) would have us believe. The reasoning of the People's Vote advocates is that as Brexit gets closer, leave voters are starting to see the error of their ways and switch sides. The polls seem to support this, though whether Brexiters were ever as unrealistic as they are sometimes painted is not clear. 

Roger Scully (of the Wales Governance Centre), who has been conducting interviews in leave areas, told me in a video interview that:

“the final major thing that we found is a strong sense amongst many of these leave voters at least that some short term difficulties with Brexit some short term costs had already been priced into their decision. They were expecting this but they still thought that in the long term Brexit would be worth it.” (9)

The poll which gave remain 53% was based on a larger than usual YouGov sample of 10,121 (if you want to see the original files they are here: (10).

If you study the results you will find that the 53-47 figure is calculated by excluding those who say they would not vote and those who say they don’t know. The figures including those are 46-40 remain, which is roughly in line with the previous month’s results (carried out with a much more regular YouGov sample of 1,725) which were 47-41 in favour of remain. In this sense, perhaps Peter Kellner, has a point when he says:

“If these findings emerged from a conventional poll of 1,000-2,000 respondents, findings relating to sub-samples should be regarded with caution.
"But this exceptionally large 10,000 sample survey contains more than 2,700 current Labour supporters, leaving no room for doubt about the size of the majority among them for a new public vote.”

His main point is that Labour supporters in the poll are more enthusiastic for a second referendum than Conservatives. This is not too much of a shock since Labour supporters voted remain in quite large numbers, but Labour’s members rejected the call for a second referendum when it was made by Owen Smith as part of the leadership campaign (11). Moreover, Labour’s 2017 Conference voted to support the current position on Brexit which is to respect the referendum result (12), although the possibility of a second referendum was not entirely ruled out (13).

If the goal of a second referendum is to reverse Brexit then it is important it is based on credible evidence. Although, Peter Kellner, a former Chief Executive of YouGov, put great stock by the size of the poll, thus arguing that it reduced the margin of error to less than 1% (meaning that the true figure could be 52-54% supporting remain) what he does not address is that YouGov samples are not random polls. According to statistical theory: 

“Determining the margin of error requires a random sample of a population”  (14)

YouGov does not randomly sample from the population, it samples from its panel. According to its website, the panel has 5 million people worldwide, and over 800,000 from the UK (15). So we know that the 10,000 people used in a YouGov survey are randomly chosen from 800,000 or so people. This makes the sample representative of those who sign up to YouGov, but not of the British public, for the simple reason that the chances of over 51 million people in the UK being selected randomly are precisely zero. 

So, talk of the size of the poll reducing the margin of error is, as I suspect Peter Kellner knows, entirely misleading. And, this is true of all YouGov surveys, despite what they might try to convince us in their FAQ’s.

So, does this mean that the poll suggesting that 53% of the UK population now support a second referendum was wrong?

Not necessarily, but it is difficult to make a claim about the entire population when you have only sampled from a narrow, self-selected sub-set of that population. The fact is that because this is not a random sample we have no way of calculating the margin of error and therefore applying a nominal +/- 3% would be highly recommended. 

The numbers in polls are often misleading, but that does not mean that polls have no use at all. They can be used to see general trends. But, of course, trends from a non-representative poll, whilst interesting, should not be taken to make misleading statements such as "The British public are moving toward..." It would only be possible to say that if the sample on which the poll was based was taken from the population as a whole.  

In the three months leading up to the EU Referendum YouGov had remain 3 points in the lead. Only in their final poll, at the end of May 2016, did the polls suggest that a remain victory was in doubt when they put the two sides neck and neck. What is really noteworthy, however, was that the ‘don’t know’s’ (for which read undecided) remained in double figures until the referendum was announced (a high of 20% in February 2016, to 14% in May). What this indicated, and continues to indicate, is that there are a number of people who are not really sure what the best option is and what appears to have happened at the referendum was that those undecideds disproportionately voted to leave. 

What also seems clear is that since the referendum the two camps have solidified. As Anthony Wells noted in a You Gov blogpost in March 2018: 

“There are still only limited signs of Bregret , with the vast majority of Leave voters still thinking it is the right decision, and the vast majority of Remain voters still believing it was wrong decision.” (16)

The lowest poll percentage for leave since the referendum was 40% (in October 2017), exactly where the polls had them on the eve of the referendum. What we might also note is that the same poll gave remain 44%, but the undecideds (10%) were in double figures for the first time since the referendum. Since October 2017 YouGov polls have been consistent with remain on 43-45%, leave on 41-43% and undecideds on 8%. (The full set of tracker results are at the bottom of this post.)

It is possible to read into these figures a very positive picture for remain who appear from the YouGov panel data to have a healthy 3% lead over leave. So, the People’s Vote campaign is not being entirely disingenuous in claiming a shift of mood, although it might be pointed out that polls in the run up to the referendum also suggested a 3% remain lead. 

For all the reasons I have rehearsed above YouGov polls cannot really be seen as representative of the population as they are not taken from the population but a self-selected panel. But putting that to one side, we can see a trend and if we use a margin of error of 3% we can see what that trend might look like. 

The apparent 47-41 advantage for remain could be 47-3 (44) to 41 +3 (44) which puts us back into eve of referendum territory. At which point how the undecideds split becomes crucial. Some recent reports assume that the undecideds will split in the same way as those who express a preference, but we know from the referendum (when compared with polls) that this was not the case in 2016 and it is a high risk strategy in 2018 if the intent of a second referendum is to change the result.

But, this brings us to the other “exciting” recent revelation which is that 112 constituencies (100 of them being Labour) would now change from leave to remain. Briefly, there are all sorts of problems with this analysis. First, the company that conducted it, Focaldata, uses a variety of statistical techniques to arrive at their conclusions. Probably because they are incredibly complicated they do not publish this on their website. But, secondly, their analysis was based on combining two YouGov polls. That is two non-random polls combined, put through a statistical ringer and then the results published as ‘fact’. 

But, even if they are right (and it is difficult to prove them wrong) what does it matter? The referendum is the aggregate of all votes counted, it doesn’t matter in which constituency you cast your vote, any more than it matters in a General Election in which street you live. 

The only purpose I can see of this particular data was to mislead Labour supporters in an attempt to put pressure on the Labour leadership to support a campaign run by people who are inherently hostile to them. 

I would return to one final point. It really doesn’t matter whether the polls are right or wrong, unless they were overwhelming, there is no reason for the Government to believe them. Furthermore, it makes no great difference to the goal of obtaining a second referendum whether it is supported by Jeremy Corbyn or not, it will be the Government who will make the decision. There does not appear, with the exception of 2 or 3 Tory MP’s, any support within Government circles for a second referendum. In which case, flawed statistics or not, it is moot. It will simply not happen.



Tracker poll data referred to in this post: