Polls versus prediction markets - Who said Brexit was a surprise?

THE list of losers from Britain’s vote to leave the European Union is long indeed, but very far down on it are evangelisers for the accuracy of prediction markets. It is an article of faith among economists that betting markets on politics provide by far the most reliable forecast of future events, easily outclassing both polls and panels of experts. Yet for the two most important political developments of 2016, and arguably of the past few years—Brexit and Donald Trump winning the Republican nomination for America’s presidency—simple polling averages have put punters to shame.

Mr Trump surged to the lead in every poll within a month of his declaring his candidacy a year ago, and never relinquished it save for a split-second tie with Ben Carson. Bettors nonetheless fancied the well-funded Jeb Bush and smooth-talking Marco Rubio for most of the lead-up to the primaries despite their lacklustre poll numbers, wrongly presuming that Mr Trump’s polling figures were bound to deflate just like those of the 2012 outsider candidates Herman Cain, Michelle Bachmann and Newt Gingrich. Even after Mr Trump had amassed an insurmountable lead in delegates in addition to dominating the polls, you could place a wager at better than even money following Ted Cruz’s ultimately meaningless win in Wisconsin.

The same is true of the Brexit vote, albeit over a shorter time period. For the vast majority of the campaign, both polls and markets had “remain” with a solid lead. When the polls shifted sharply towards “leave” in early June, bettors moved in the same direction, but not enough—never once did “leave” come close to taking the lead. And after a few “remain”-friendly surveys shortly before the vote moved the polling average close to a dead heat, markets took that as a clear sign that “remain” would coast to a comfortable victory. On election day, they priced about an 85% likelihood that Britain would stay in the EU.

Such errors are hard to fathom. Pollsters have access to only one source: how their respondents say they plan to vote at any one moment. In contrast, bettors in prediction markets have full knowledge of all public polls, plus other valuable types of information such as fundraising, endorsements, media coverage, insiders trading on non-public knowledge and the like. Unlike polls, prediction markets weight individuals’ beliefs by conviction as well as frequency: someone who is really sure about an event is likely to wager more on it than someone who is “just taking a punt”.

How did the wisdom of crowds fail so spectacularly? One theory holds that the Brexit market was swayed by a small number of big bets by optimistic “remain” voters, who tended to be richer than those who supported “leave” (indeed, Ladbrokes, a bookmaker, has said that the majority of individual wagers were placed for “leave”). But while political-betting markets could conceivably be small enough to demonstrate such inefficiencies, currency markets most certainly are not, and they displayed the same pattern as the bookies.

Instead, the explanation probably lies in the familiar litany of cognitive biases that lead people astray despite their best efforts to be rational. Historically prediction markets, like horse races, have tended to demonstrate favourite-longshot bias—overestimating the chances of improbable events and underestimating those of likely ones. But betting on the Brexit referendum seems to have displayed the opposite pattern: under-pricing kurtosis or fat-tail risk, and thus the chances of an unlikely but devastating “black-swan” event. People who found it unfathomable that Britain could vote to leave, primarily because such an event had never happened before, probably in turn succumbed to confirmation bias: placing more weight on recently released polls favouring “remain” than on the similar number of surveys backing “leave”. They may have also fallen victim to the “availability heuristic”, presuming that the EU vote was likely to resemble that of Scotland’s 2014 independence referendum (where the status quo won handily even though the polls showed a tight race) simply because that precedent occurred so recently in the same country.

Mr Trump’s nomination and the Brexit vote are only two events among the thousands that have been wagered on. If the markets are doing their jobs, then one-in-five shots should come in pretty often. The Bayesian approach to interpreting the world—starting with a prior belief, and then updating it to reflect new information—still has much to recommend itself over the frequentist philosophy, which would simply take a polling average as gospel. But Bayesianism requires both sound priors and their rapid adjustment when new evidence is overwhelming, and it seems that in these two cases investors clung to their priors far too closely. For all the talk of a polling crisis and a handful of prominent misfires, they did an outstanding job of predicting these supposedly unthinkable events. It’s fine to disregard what many thousands of people are telling pollsters they plan to do, but in the absence of compelling evidence to the contrary, it’s also probably a good starting point to assume they’re telling the truth.

source The Economist, Friday, June 24, 2016

A little thought experiment - How to be fair to the pro-EU Scotland

ALEX SALMOND, the ex-leader of the Scottish National Party, has floated an interesting idea: could Scotland "inherit" Britain's EU membership, now that there has been a vote to leave? In the referendum Scotland turned out to be very pro-EU, whereas most of the rest of Britain leant towards Brexit. That has sparked calls for another Scottish independence referendum, which to many seem justified: why should Scotland be dragged out of Europe by England? It's probably fair to say that support for independence could now be the majority in Scotland. 

The BBC, which reported Mr Salmond's comments, explores the question he raises in a conventional manner. "If Scotland were to hold a second referendum, and become independent," they say, "it could apply to become a member of the EU in the usual way." In order for an independent Scotland to join, the other EU states would have to agree. 

However, as people who followed the 2014 Scottish independence referendum will remember, the process of Scotland going from independence to EU membership is extremely challenging. Countries like Spain, which are contending with their own separatist movements (eg, the Basque region and Catalonia), would try to block Scotland's accession. So Scotland could be left in the position of being independent but not in the EU, which is definitely not what it wants.

So, is there a way that there could be a) a vote for Scottish independence and b) a vote that ensured that Scotland could be part of EU in the event? How about the following:

(For the sake of convenience in this analysis, ignore Northern Ireland. Assume Scotland definitely wants to be independent of Britain.) David Cameron, who has just announced that he will be resigning later in the year, calls a referendum on Scottish independence just before he quits. To many people, this will seem unobjectionable.

However, there is a catch. Rather than Scottish people voting to become independent from Britain, the referendum is structured such that non-Scottish British people vote to become independent from Britain. In other words, if non-Scottish people vote in favour of "Scottish independence", then what really happens is that Wales and England become independent from the United Kingdom. At that point the "UK" would consist of just one country, Scotland. And so Scotland would remain in the EU. 

This approach has some merits. For one, in the event of a "yes" vote Scotland could become independent from the rest of the UK and simultaneously not drop out of the European Union. In addition, it would satisfy demands in England and Wales for independence from the European Union (because the independent country, England/Wales, would no longer be part of the EU, an analogous situation as would have emerged if Scotland had voted for independence in 2014).

There are a few obvious objections. Mr Cameron may not have the political authority to schedule another referendum. Breaking up this union is not popular in England and Wales: they may well vote to remain in the United Kingdom, thus denying Scottish independence (but in the process choosing to keep Wales and England in the EU). However, lots of people would agree to the idea of there being another Scottish referendum. And this type of Scottish referendum is fair to both Scotland and England/Wales, in that it allows both parties ultimately to get what they want.

So there. Perhaps.

source The Economist, Friday, June 24, 2016

The Labour Party and Brexit - The culpability of Jeremy Corbyn

Mr Corbyn’s weak campaign to remain in the EU leaves him scorned by MPs and northern voters alike

Just as the prime minister, David Cameron, has already announced his resignation as a result of the referendum vote, many are now questioning whether Jeremy Corbyn, the leader of the Labour Party, can survive.

The Labour leadership was in favour of remaining in the EU, yet the vote was lost in the Labour heartlands of the Midlands and north of England. In the north-east, a strong Labour area, the Remain vote was much weaker than expected. Many furious Labour politicians blame this squarely on Mr Corbyn. “Jeremy Corbyn has got to take responsibility,” said Angela Smith, a former shadow minister. “He should consider his position. He’s shown insufficient leadership.” Two Labour MPs, Margaret Hodge and Ann Coffey, have sent a letter to John Cryer, chair of the Parliamentary Labour Party (PLP), submitting a motion of no confidence in Mr Corbyn to be discussed at the next meeting of the PLP on June 27th. If Mr Cryer allows this motion to be debated, it could be followed by a secret ballot of Labour MPs the following day. This would be a strong expression of the lack of confidence in Mr Corbyn, even if it has no constitutional force to unseat him.

Labour critics accuse their leader of being at best half-committed to the cause, of doing much too little much too late, and of ignoring the main issue of concern to traditional Labour voters: immigration. The former Labour cabinet minister Lord Peter Mandelson put the boot into Mr Corbyn, telling an interviewer that he had been “travelling through the referendum with the handbrake on” and that he had failed to support his senior colleagues on the campaign trail. Furthermore, argued Lord Mandelson, his palpable failings during the referendum campaign were evidence of a more profound inability to “project his message”. Perhaps Mr Corbyn doesn’t have what it takes, he mused.

Lord Mandelson’s comments will be dismissed by Mr Corbyn’s supporters as just more grousing from the right wing of the party. But they reflect a wider anger and frustration with Mr Corbyn’s leadership. “Jeremy hasn’t pulled his weight”, said one Labour MP. “The EU referendum simply shone a light on how utterly out of touch Corbyn and [John] McDonnell [the shadow chancellor] are with so many traditional Labour voters outside of London.” Another MP said: “Even people who supported him have seen he is not up to it. He can’t motivate Labour voters, let alone persuade anyone else.” Labour MPs were amazed that a sheet of “talking points” prepared for them by head office on how to respond to the defeat asked them to claim that Corbyn had “led from the front” during the campaign, and that he was now “far closer to the centre of gravity of the British public than other politicians.”

Scorn from his backbenchers is not sufficient to displace Mr Corbyn. His position as Labour leader has always rested on his popularity among the party’s paid-up supporters rather than its MPs. The left-winger was elected in 2015 with an overwhelming mandate, winning 59.5% of the vote of party members and affiliated supporters. They may yet stand by their man.

All that would prove, however, is how far out of touch the activists are with Labour voters, especially in the north of England. This was a point made by John Mann, the MP for Bassetlaw in the east Midlands, one of the few Labour MPs who campaigned for a Brexit. “Labour has gone wrong,” he said immediately after the result, “by not being in touch with the voters.” Mr Mann’s critique must be particularly worrying for Mr Corbyn, as it comes not from his old foes on the right of the party, like the Blairite Lord Mandelson, but from the trade-union core. People like Mr Mann view Mr Corbyn’s brand of urban, cosmopolitan left-wing protest politics as thoroughly divorced from the Labour heartlands. The referendum has merely confirmed their worst fears.

source The Economist, Friday, June 24, 2016