Polls versus prediction markets - Who said Brexit was a surprise?

THE list of losers from Britain’s vote to leave the European Union is long indeed, but very far down on it are evangelisers for the accuracy of prediction markets. It is an article of faith among economists that betting markets on politics provide by far the most reliable forecast of future events, easily outclassing both polls and panels of experts. Yet for the two most important political developments of 2016, and arguably of the past few years—Brexit and Donald Trump winning the Republican nomination for America’s presidency—simple polling averages have put punters to shame.

Mr Trump surged to the lead in every poll within a month of his declaring his candidacy a year ago, and never relinquished it save for a split-second tie with Ben Carson. Bettors nonetheless fancied the well-funded Jeb Bush and smooth-talking Marco Rubio for most of the lead-up to the primaries despite their lacklustre poll numbers, wrongly presuming that Mr Trump’s polling figures were bound to deflate just like those of the 2012 outsider candidates Herman Cain, Michelle Bachmann and Newt Gingrich. Even after Mr Trump had amassed an insurmountable lead in delegates in addition to dominating the polls, you could place a wager at better than even money following Ted Cruz’s ultimately meaningless win in Wisconsin.

The same is true of the Brexit vote, albeit over a shorter time period. For the vast majority of the campaign, both polls and markets had “remain” with a solid lead. When the polls shifted sharply towards “leave” in early June, bettors moved in the same direction, but not enough—never once did “leave” come close to taking the lead. And after a few “remain”-friendly surveys shortly before the vote moved the polling average close to a dead heat, markets took that as a clear sign that “remain” would coast to a comfortable victory. On election day, they priced about an 85% likelihood that Britain would stay in the EU.

Such errors are hard to fathom. Pollsters have access to only one source: how their respondents say they plan to vote at any one moment. In contrast, bettors in prediction markets have full knowledge of all public polls, plus other valuable types of information such as fundraising, endorsements, media coverage, insiders trading on non-public knowledge and the like. Unlike polls, prediction markets weight individuals’ beliefs by conviction as well as frequency: someone who is really sure about an event is likely to wager more on it than someone who is “just taking a punt”.

How did the wisdom of crowds fail so spectacularly? One theory holds that the Brexit market was swayed by a small number of big bets by optimistic “remain” voters, who tended to be richer than those who supported “leave” (indeed, Ladbrokes, a bookmaker, has said that the majority of individual wagers were placed for “leave”). But while political-betting markets could conceivably be small enough to demonstrate such inefficiencies, currency markets most certainly are not, and they displayed the same pattern as the bookies.

Instead, the explanation probably lies in the familiar litany of cognitive biases that lead people astray despite their best efforts to be rational. Historically prediction markets, like horse races, have tended to demonstrate favourite-longshot bias—overestimating the chances of improbable events and underestimating those of likely ones. But betting on the Brexit referendum seems to have displayed the opposite pattern: under-pricing kurtosis or fat-tail risk, and thus the chances of an unlikely but devastating “black-swan” event. People who found it unfathomable that Britain could vote to leave, primarily because such an event had never happened before, probably in turn succumbed to confirmation bias: placing more weight on recently released polls favouring “remain” than on the similar number of surveys backing “leave”. They may have also fallen victim to the “availability heuristic”, presuming that the EU vote was likely to resemble that of Scotland’s 2014 independence referendum (where the status quo won handily even though the polls showed a tight race) simply because that precedent occurred so recently in the same country.

Mr Trump’s nomination and the Brexit vote are only two events among the thousands that have been wagered on. If the markets are doing their jobs, then one-in-five shots should come in pretty often. The Bayesian approach to interpreting the world—starting with a prior belief, and then updating it to reflect new information—still has much to recommend itself over the frequentist philosophy, which would simply take a polling average as gospel. But Bayesianism requires both sound priors and their rapid adjustment when new evidence is overwhelming, and it seems that in these two cases investors clung to their priors far too closely. For all the talk of a polling crisis and a handful of prominent misfires, they did an outstanding job of predicting these supposedly unthinkable events. It’s fine to disregard what many thousands of people are telling pollsters they plan to do, but in the absence of compelling evidence to the contrary, it’s also probably a good starting point to assume they’re telling the truth.

source The Economist, Friday, June 24, 2016