Opinion

Felix Salmon

When quants tell stories

By Felix Salmon
November 7, 2012

The dominant narrative, the day after the presidential election, is the triumph of the quants. As Simon Jackman notes, essentially every single poll-averaging quant — Jackman himself, Drew Linzer, Sam Wang, you name it — managed to call every single state plus the presidential election: an astonishing 51/51 success rate.

That was part skill and part luck, as all such things are. Here’s the final electoral-vote probability distribution from Nate Silver, the most famous of the quants:

Obama ended up winning 332 electoral votes, assuming that he ends up winning Florida; that was the single most likely outcome in Silver’s model, with a probability of just over 20%. But the mode outcome isn’t the median outcome: the official Nate Silver forecast was that Obama would get 313 votes. Roughly 80% of the time, under Silver’s model, he would have ended up calling at least one state wrong. So Silver did get lucky.

Silver is the most visible of the quants, partly because of his perch at the NYT, partly because he has a new book out, and partly because he’s very good at taking his complex mathematical model and turning it into bite-sized English-language blog posts. If you think that the value of Nate Silver is in the model, you’re missing the most important part: there are lots of people with models, and most of those models are pretty similar to each other. The thing which sets Silver apart from the rest is that he can write: he can take a model and turn it into a narrative, walking his readers through to his conclusions.

Which brings me to Michael Scherer’s great story about the Obama-campaign quants: the people who A/B tested everything and who built an astonishingly formidable ground game. Rather than just blanketing the airwaves with ads they thought were good, the Obama campaign was constantly testing, quantifying, and targeting.

At heart, the campaign was marrying quantitative skills with storytelling, to unbeatable effect. Which stories should the campaign tell, to any given group of people? How should it tell those stories? And who should it get to deliver those stories? The database answered all those questions:

Cash raised online came through an intricate, metric-driven e-mail campaign in which dozens of fundraising appeals went out each day. Here again, data collection and analysis were paramount. Many of the e-mails sent to supporters were just tests, with different subject lines, senders and messages. Inside the campaign, there were office pools on which combination would raise the most money, and often the pools got it wrong…

In the final weeks of the campaign, people who had downloaded an app were sent messages with pictures of their friends in swing states. They were told to click a button to automatically urge those targeted voters to take certain actions, such as registering to vote, voting early or getting to the polls. The campaign found that roughly 1 in 5 people contacted by a Facebook pal acted on the request, in large part because the message came from someone they knew…

“We were able to put our target voters through some really complicated modeling, to say, O.K., if Miami-Dade women under 35 are the targets, [here is] how to reach them,” said one official. As a result, the campaign bought ads to air during unconventional programming, like Sons of Anarchy, The Walking Dead and Don’t Trust the B—- in Apt. 23, skirting the traditional route of buying ads next to local news programming. How much more efficient was the Obama campaign of 2012 than 2008 at ad buying? Chicago has a number for that: “On TV we were able to buy 14% more efficiently … to make sure we were talking to our persuadable voters,” the same official said.

The thing that Silver and the Obama campaign have in common, then, is that they used their databases to tell stories. Or, more to the point, their databases and models were used so that Americans could tell stories to each other. Silver’s site became a virtual watercooler, especially towards the end of the campaign — a place where people would gather to talk about what was possible and what was likely. Nate’s voice helped to guide the discussions, but the real reason that he got such astonishing traffic was not that people wanted to read what he was writing, so much as that people were using his model as a framework within which to hold their own idiosyncratic discussions.

Similarly, the Obama campaign put enormous effort into making sure that when phone calls were made, the right people were talking to each other. Scripts are bad; one-to-one human connections are good. This is the age of Facebook, where big data meets the social graph: I’m sure the Romney campaign had a big database too, but it lacked the same storytelling ability, and lacked the same degree of insight into the possible connections that could be drawn within the universe of supporters and potential supporters.

Here’s another important part of the Scherer article:

For all the praise Obama’s team won in 2008 for its high-tech wizardry, its success masked a huge weakness: too many databases. Back then, volunteers making phone calls through the Obama website were working off lists that differed from the lists used by callers in the campaign office. Get-out-the-vote lists were never reconciled with fundraising lists. It was like the FBI and the CIA before 9/11: the two camps never shared data. “We analyzed very early that the problem in Democratic politics was you had databases all over the place,” said one of the officials. “None of them talked to each other.” So over the first 18 months, the campaign started over, creating a single massive system that could merge the information collected from pollsters, fundraisers, field workers and consumer databases as well as social-media and mobile contacts with the main Democratic voter files in the swing states.

This gave the Obama campaign a massive advantage over the Romney campaign. The Obama campaigns data was centralized and coordinated, while the Romney campaign relied in large part on SuperPACs which by law could not have access to the central database and could not be coordinated.

SuperPACs are dumb money. Their cash can almost never be effectively spent, because they’re not on the inside of the campaign. What’s more, because they’re not official campaigns, they always pay top dollar for their TV ad spots, rather than the discounted rates that stations are forced to offer to candidates. The Obama campaign determined, at various points, that if they approached potential donors with the message that Romney had a fundraising lead, that would help Obama raise more money for his own campaign. But the truth was that Romney’s fundraising lead was never particularly useful, because it wasn’t married to a coherent strategy and database. If anything, it just helped Obama raise more cash.

Obama is never going to run another campaign, so the advantage that Obama had over Romney does not necessarily mean that in the 2016 race the Democratic candidate is going to have a similar advantage. But we do know that the Democrats have the technology. And, at least for the time being, the Republicans don’t.

Comments
8 comments so far | RSS Comments RSS

Nate Silver was the best and most innovative of the quants back when he was developing the PECOTA system for baseball projection. He revolutionized the field!

Hardly shocking that he is having a similar impact in projecting political races.

Posted by TFF | Report as abusive
 

Felix, I’ve been writing a bit this morning about whether Fivethirtyeight reflects a “risk-free” rate compared to Intrade odds, and wondered whether you think that’s an idea that is likely to gain much traction in the gambling/hedge fund worlds. I use the phrase “risk-free” with all possible associations from the past few years, of course.

Posted by GregorHunter | Report as abusive
 

I think Nate Silver is great, predominately for the idea that facts/reality matters.
But for all the slicing and dicing of demographics, one thing can get missed. What the candidates believe – if Obama espoused Romney’s positions, and Romney espoused Obama’s positions (of course, on fundamental questions like serving their wall street masters, they are in complete agreement), we would now have president Romney.
Unfortunately, instead of examining their policies, the republicans will attribute their loss to “reverse racism” – they will either have to learn that being anti gay, representing the 0.1%, and that ineptly managing government (saying you are for a balanced budget when you are defintely not) is a losing stratgy, or go extinct.

Posted by fresnodan | Report as abusive
 

@fresnodan, by “reverse racism”, do you mean “voting against racist candidates and racist policies”? Shouldn’t that be called “anti-racism”?

Romney lost the Hispanic vote due to espousing anti-Hispanic policies. No residency for somebody raised in the US from childhood? And Romney lost my vote when he declared (to a room filled with rich white guys), “I don’t care about those people.” Meaning anybody who isn’t rich, white, and male.

Might have ultimately voted against him anyways. You cannot simultaneously promise to cut taxes and balance the budget while protecting Social Security and Medicare. The numbers don’t add up. SOMEWHERE in that list of promises is a BIG LIE.

Posted by TFF | Report as abusive
 

When reuters/ipsos was all over how close this presidential race was who did they sample, one guy in Wyoming and a woman in Massachusetts? Other than generating headlines the pre-election media polls really sucked this time around. Why?

Posted by Woltmann | Report as abusive
 

@Woltmann, the national polls were on target — Obama and Romney finished with very similar vote counts.

However the president isn’t elected by popular vote, the president is elected by the electoral college. When Romney won states, he won by very large margins. When he lost states, he lost by narrower margins.

Posted by TFF | Report as abusive
 

@TFF, you’re right, reuter/ipsos probably didn’t know about that whole electoral college thing. They probably thought the president was elected by popular vote too ..

Posted by Woltmann | Report as abusive
 

@Woltmann, the whole concept of a national poll is flawed for exactly that reason. Anybody who bothers to conduct a national poll is clearly not interested in who will win the election.

Posted by TFF | Report as abusive
 

Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/
  •