Opinion

The Great Debate

May the odds be ever in your favor

By Chadwick Matlin
October 24, 2012

Editor’s note: This piece was originally written for Tomorrow Magazine, whose first issue comes out this month. The article is being republished with permission.

On May 20, 2011, John Delaney awoke 550 meters from the summit of Mount Everest. Delaney, who founded Intrade, a website for those who love to predict the future, had been trying to get to the top of the world for years. His company invited users to bet on the news: Customers would calculate probabilities, assess risk, make a wager. On Everest, Delaney was doing much the same.

This close to the summit, he was on an area of the mountain known as the death zone, where the atmosphere is about three times thinner than at sea level. Cerebral and pulmonary edemas—the leaking of fluid to the brain and heart—are increasingly likely at that altitude. Still, between 1921 and 2006, just 94 people died at this stage of the climb—above 8,000 meters, but before the summit. Altogether, Everest claimed 192 climbers’ lives in that 86-year span, 1.3 percent of those who attempted a climb. The odds that Delaney would succumb to the mountain were low.

Back in Ireland, three days earlier, his wife, Orla, had given birth to a premature baby, though only 6 percent of babies born in Ireland are premature. She named her Hope.

But Delaney was too far from Orla and too close to the summit to be notified. At 7:30 p.m., he set out from camp, 8,300 meters above sea level. According to forecasts, the temperature was -7 degrees Fahrenheit, and winds blew as fast as 40 miles an hour.

Each of the eight climbers in the group had their own sherpa and oxygen supply. At 1:45 a.m., 50 meters from the summit, Delaney began to struggle. A guide came down from the summit and, with the help of sherpas, began to escort him back down the mountain. At 4:30 a.m., 100 meters from the summit of Everest, the guides pronounced Delaney dead. He would never know Hope.

“John and I were both very much aware of the risks involved,” Orla said at his funeral. Everest deaths were not unheard of—the statistics are publicly available. The Delaneys knew that something could go wrong; they just assumed that it wouldn’t. Tragically, they made the wrong prediction. The man who made his living taking bets had succumbed to his own.

***

Delaney left behind his wife, three kids, and a company that has changed the way political junkies obsess about elections. Intrade, founded as a small, unassuming betting market in Dublin, became part of a new trend in American politics. Treating politicians like stocks, Intrade’s website offers markets for people to invest in their conviction that Mitt Romney would win the 2012 Republican presidential nomination. The result? Quantified public perception of the race—“conventional wisdom, plus” as one academic told me.

Intrade wasn’t the first to offer a political prediction market, but it was the savviest. Opened in 2001 as an online sports-betting market, the company quickly transitioned to politics when Delaney saw demand for its prop political bets. It had competition—Iowa Electronic Markets and BetFair, among others—but by opening itself up to the media and researchers, it became for political betting markets what Kleenex is to facial tissue. It proved irresistible to political journalists, for whom the horse race is stock-in-trade.

As Intrade came into its own in 2008, a geek emerged from the cornfields of baseball statistics to offer an alternative method of predicting the future. He was anonymous at first, one of the masses in the scrum at liberal blog DailyKos, username: poblano. But then he started his own site, fivethirtyeight.com, and outed himself. His name was Nate Silver, and he was ready to do for politics what statistician Bill James had done for baseball: Make statistics a force to be reckoned with.

Silver created models that predicted the winners of the 2008 campaign, just like Intrade. His predictions, though didn’t rely on the wisdom of crowds, but on calculations. Amazingly, they were right—sometimes even better than the polls they were based on. Silver became an oracle.

Four years later, Intrade and Silver are further entrenched in the national political conversation. Pundits cite Intrade market prices to prove their points, even if the market they cite is hopelessly flawed. In the first nine months of 2012, more than $124 million was traded on Intrade’s political markets. By August, the number of political bets had surpassed its 2008 peak by 2.2 million. By 2010, meanwhile, editors at The New York Times were so impressed with Silver they hired him to bolster the paper’s election coverage; Silver also signed a book contract with a reported $700,000 advance. The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t came out in late September.

Silver and Intrade are vastly different touchstones in our new predictive era. One is democratic, the other technocratic. And yet their success springs from the same well: a human desire to forecast the unknown, and a modern desire to do it with data. Good thing we’re overflowing with it.

***

Political prediction is a renaissance, not a revolution. American political observers were building betting pools as early as the Lincoln Administration, according to economists Paul Rohde and Koleman Strumpf. By the 1890s, Wall Street supervised the betting, and newspapers reported the latest odds in the next edition. “In presidential races in 1896, 1900, 1904, 1916, and 1924, The New York Times, Sun, and World provided nearly daily quotes from early October until Election Day,” Rohde and Strumpf write. One hundred years before Nate Silver, the Times was reporting on election odds.

These markets were proto-Intrades. Adjusted for inflation, tens of millions of dollars were invested, topping out at an absurd $211 million (in today’s dollars) for the down-to-the-wire Wilson-Hughes race of 1916. Moreover, the markets were quite predictive. “The capacity of the betting markets to aggregate information is all the more remarkable given the absence of scientific polls,” Rohde and Strumpf write. Crowd-sourced wisdom didn’t need formalized data—just gut feel.

Justin Wolfers is an economist at the University of Pennsylvania whose outspoken support of prediction markets has made him the public face of the field. To him, these markets are a practical use of economic principle. There are buyers, there are sellers, and their exchanges tell us about the world, no matter the era in which the action is taking place. “Prediction markets were extremely accurate 100 years ago,” Wolfers says. “The HTML around them have changed, but the underlying economics haven’t.” Likewise, Silver’s models aren’t novel. They’ve been hiding in academia for decades. “Nate’s doing a few things that are clever, many of which were in political science literature, all of which make sense,” says political scientist Simon Jackman, who developed prediction models while Silver was still in college.

If the Silvers and Intrades of the past faded, why have they reemerged, and why do they feel so new? By World War II, betting markets had all but disappeared. Spooked by the potential for people to rig the election with their bets, New York’s state government cracked down on electoral gambling. By the time Gallup polls established themselves in the late 1930s, electoral bets were out of the papers. They wouldn’t reemerge with any popularity until the 21st century.

Jackman has a theory about the undulating rise and fall of political betting markets: People only bet their money when they think they have a chance of winning. When you’re betting on information, you want to make sure you have just as much as everybody else. In the early 20th century, betting markets were “democratic because there wasn’t much [information],” he says. Once scientific polling began, information became undemocratic. You never knew if somebody had access to a poll you didn’t, so why take the chance?

Now markets are once again open for all “because we’re drowning,” Jackman says. Drowning in information, in data, in input. There’s enough noise for all of us to sort through and find our own signal. The web lets us all access the same public opinion polls, subscribe to the same breaking news Twitter accounts, and see the same GIFs of Barack Obama slow-jamming the news. In the gaping maw of a nonstop news cycle, we’re finding more and more data (I’m using that term loosely) to fill the void.

“We face danger whenever information growth outpaces our understanding of how to process it,” Silver writes in his book. If only there were someone who was willing to demystify the polling, tell us who to trust and who to dismiss—and, well, just tell us who’s going to win.

***

Silver doesn’t think what he does for a living is complex, which is why he compares it to rocket science. “Even rocket science is trying to coordinate a bunch of relatively simple things,” he told me.

Silver’s list of simple things includes aggregating every poll that’s released, weighting them according to their trustworthiness (defined by past performance), combining those polls with various indicators—economic statistics, a candidate’s incumbency, the state’s demographics—and using it all to calculate who’s going to win the presidency. As he said, simple.

Silver graduated from the University of Chicago with a degree in economics in 2000, then whiled away a few years as an economic consultant for an accounting firm. Bored, he surreptitiously created a spreadsheet that forecast baseball players’ futures based on statistical measurements of similar players. After it was purchased by Baseball Prospectus, a clearinghouse for baseball stats, he quit his job to play online poker, at one point netting $400,000.

He first caught the attention of political junkies when, still anonymous on DailyKos, he predicted the results of the 2008 Democratic primaries not by interpreting polls, but demographics and past votes. His model predicted Barack Obama would win by 17 points in North Carolina and lose by 2 points in Indiana; pundits predicted that he’d win Carolina by about 7 and lose Indiana by about 4. He won Carolina by 14 and lost Indiana by 2. Silver had proved his bona fides.

Since then, his method has evolved. For a few cycles, sites like Pollster.com and RealClearPolitics averaged the glut of polls to present a reliable snapshot of the race. But it was just that—a snapshot. Silver wanted to do more. He wanted to take pictures of the future. After the North Carolina experiment, he started to blend economic and demographic indicators with polls to create a new type of prediction model.

Mark Blumenthal, the founding editor of Pollster.com, was, in many ways, the Silver of 2004, a guy who came out of nowhere to help the general public better understand the quantitative side of politics. But now he’s concerned about where Silver has sent the discipline.

“I don’t want to make it sound like I think Silver’s evil, though with a couple of beers I probably could,” he told me, admitting that as a competitor, he might be biased. All of Silver’s extras, Blumenthal argues, don’t do much more than Pollster.com’s average because the base of Silver’s model are the same polls. Nevertheless, Blumenthal says with a resigned laugh, “it’s certainly sophisticated. It’s certainly complex.”

Blumenthal is concerned that Silver is mixing punditry with his polls, and that readers aren’t savvy enough to see it that way. “People out there email that the best poll out there is Nate Silver’s FiveThirtyEight, but it’s not a poll,” he told me. Silver, remember, decides what goes in his model and what stays out. That comes with the territory, really—the model can’t make itself. It needs a creator. “The numbers have no way of speaking for themselves. We speak for them,” Silver writes in his book. “It is when we deny our role in the process that the odds of failure rise.”

The details of Silver’s models are crucial, but most readers can’t be bothered with them. (Silver told me he wishes he could provide complete methodologies, he just doesn’t have the time.) For the mainstream reader, Silver’s final probability score is all that’s remembered—it doesn’t matter what it’s made of. Still, even Sam Wang, a Princeton neuroscientist and political modeler who Silver describes as “one of my most frequent critics,” says Silver’s quantified analysis “is missing from most political commentary, which is statistically not sophisticated. The mass media needs more people like him.”

***

In the wake of John Delaney’s death, Intrade has no icon to rival Silver. It’s just four people in a Dublin office, managing a system that draws more than 2,000 trades a day. It hasn’t stopped growing since it transitioned to news bets. In 2004, 2 million political shares were traded; in 2008, 8 million; with two months to go in the 2012 presidential election, there were 12 million.

Intrade’s most-trafficked markets, by far, are the ones about U.S. politics, even though U.S. banks don’t allow customers to bet on Intrade markets. If an upstanding American wanted to wager on whom Mitt Romney would choose as his vice president in July, he’d need to find a workaround, often by using a wire transfer to a foreign financial institution. Nevertheless, three-quarters of new accounts originate from the U.S.

The problem is that Intrade markets can sometimes be, well, totally useless. Barry Ritholtz, a financial writer, has been on a crusade against the platform for years because of its willingness to let people predict things they have no ability to predict. “The traders as a group have no correlation to the decision-makers,” he wrote on his blog after Intrade confidently forecast that the Supreme Court would rule Obamacare’s individual mandate unconstitutional. Intrade is also afflicted by a longshot bias, a phenomenon common to all markets in which bettors overvalue the chances of a dark horse. And sometimes, there simply aren’t enough traders to produce a full picture of events. The market on the Missouri Senate race between Claire McCaskill and Todd Akin, for example, was woefully inactive until Akin made his “legitimate rape” comment. All that means is that sometimes, Intrade makes the wrong forecast.

“Every time something happens where an underdog wins I get calls from six journalists saying Intrade said it wouldn’t happen,” says Wolfers, the economist with the prediction penchant. But, Wolfers argues, prediction markets are probability markets. The upsets aren’t jarring, they’re just black swans, unlikely events come to fruition. If Obama’s stock is trading at 60 out of 100, say, that’s not an ironclad decree about who’s going to win. It’s a probability, just like Silver’s calculations. Obama’s 60 percent stock really means he’s going to win the election 60 times out of 100.

David Rothschild, an economist who specializes in data-based forecasting, compared Intrade and Silver’s performances in 2008 and found Intrade was just as good, if not slightly better, than Silver in close races. In the presidential race, Intrade’s predictions were regularly more accurate than Silver’s weeks earlier in the cycle.

***

That doesn’t mean Rothschild thinks one is definitively better than the other. “It doesn’t make any sense to be a partisan of one set of data or another,” he told me over a beer in Manhattan. Intrade can be an invaluable snapshot of rapid reaction to breaking news, while Silver performs the kinds of calculations no Intrade investor has the time, skills, or assets to do and presents them in a way that people can easily understand. Together, they’re a buddy-cop squad from the future.

But both share a weakness: their trust in polls. Intrade and Silver rely on the same set of public-opinion surveys to give a snapshot of today’s opinions. Pollsters, though, are increasingly threatened by budget cuts and the high cost of cellphone polling.* If the accuracy of their surveys is weak, then the machines’ final product will be, too. “I live in fear but also the recognition that one year we’re going to have the polls way off,” Silver says. At some point a future president is going to channel Harry Truman, gleefully holding up an iPad with FiveThirtyEight’s mistaken forecast on display.

***

The nation’s prospects seem far more uncertain than John Delaney’s Everest climb: Unemployment and student debt, a never-ending culture clash at home and abroad, climate change. Uncertainty is literally blamed for our economic problems. Amid all this, we look for signs—data—that can help us predict what might happen. What might happen politically matters more than nearly anything else.

In his book The Victory Lab: The Secret Science of Winning Campaigns, Sasha Issenberg writes that campaigns have been doing this for years, using data to calculate how likely a person is to vote or canvas for a candidate given past behavior. They aren’t trying to divine the future, they’re trying to connect the dots. “There is less ambiguity,” he says about the campaign efforts, “than there is in predicting something that’s going to happen.”

Nevertheless, Issenberg reports that the Obama campaign reached out to Silver in 2008 for “a little external validation that what we were seeing is what was actually going on,” as one adviser put it. No matter how democratic information becomes, experts like Silver are still in demand to synthesize it for the masses. The free flow of probability in the wild can be senseless and terrifying—look at Intrade’s volatility after a big news day, or Delaney’s tragic fate.

Only on rare occasions, like Election Day, do we allow the voice of the many to crowd out the oracular few. That’s one reason, perhaps, we’re so focused on trying to understand the results in advance. But when the voters turn out, they’re not predicting anything, they’re deciding—until next season, when the forecasters start their climbs anew.

—————————–

To read more pieces like this in Tomorrow Magazine, a new magazine from the former editors of GOOD Magazine, you can buy the first issue here.

CORRECTION: Because of an editing error, this piece originally referred to Pollster.com’s budget being cut. It should have read that pollsters budgets are being cut.

Comments
One comment so far | RSS Comments RSS

Hope wasn’t a premature baby. She was born 3/4 days ahead of schedule.

Posted by APKH | Report as abusive
 

Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/
  •