When quants tell stories

November 7, 2012

The dominant narrative, the day after the presidential election, is the triumph of the quants. As Simon Jackman notes, essentially every single poll-averaging quant — Jackman himself, Drew Linzer, Sam Wang, you name it — managed to call every single state plus the presidential election: an astonishing 51/51 success rate.

That was part skill and part luck, as all such things are. Here’s the final electoral-vote probability distribution from Nate Silver, the most famous of the quants:

Obama ended up winning 332 electoral votes, assuming that he ends up winning Florida; that was the single most likely outcome in Silver’s model, with a probability of just over 20%. But the mode outcome isn’t the median outcome: the official Nate Silver forecast was that Obama would get 313 votes. Roughly 80% of the time, under Silver’s model, he would have ended up calling at least one state wrong. So Silver did get lucky.

Silver is the most visible of the quants, partly because of his perch at the NYT, partly because he has a new book out, and partly because he’s very good at taking his complex mathematical model and turning it into bite-sized English-language blog posts. If you think that the value of Nate Silver is in the model, you’re missing the most important part: there are lots of people with models, and most of those models are pretty similar to each other. The thing which sets Silver apart from the rest is that he can write: he can take a model and turn it into a narrative, walking his readers through to his conclusions.

Which brings me to Michael Scherer’s great story about the Obama-campaign quants: the people who A/B tested everything and who built an astonishingly formidable ground game. Rather than just blanketing the airwaves with ads they thought were good, the Obama campaign was constantly testing, quantifying, and targeting.

At heart, the campaign was marrying quantitative skills with storytelling, to unbeatable effect. Which stories should the campaign tell, to any given group of people? How should it tell those stories? And who should it get to deliver those stories? The database answered all those questions:

Cash raised online came through an intricate, metric-driven e-mail campaign in which dozens of fundraising appeals went out each day. Here again, data collection and analysis were paramount. Many of the e-mails sent to supporters were just tests, with different subject lines, senders and messages. Inside the campaign, there were office pools on which combination would raise the most money, and often the pools got it wrong…

In the final weeks of the campaign, people who had downloaded an app were sent messages with pictures of their friends in swing states. They were told to click a button to automatically urge those targeted voters to take certain actions, such as registering to vote, voting early or getting to the polls. The campaign found that roughly 1 in 5 people contacted by a Facebook pal acted on the request, in large part because the message came from someone they knew…

“We were able to put our target voters through some really complicated modeling, to say, O.K., if Miami-Dade women under 35 are the targets, [here is] how to reach them,” said one official. As a result, the campaign bought ads to air during unconventional programming, like Sons of Anarchy, The Walking Dead and Don’t Trust the B—- in Apt. 23, skirting the traditional route of buying ads next to local news programming. How much more efficient was the Obama campaign of 2012 than 2008 at ad buying? Chicago has a number for that: “On TV we were able to buy 14% more efficiently … to make sure we were talking to our persuadable voters,” the same official said.

The thing that Silver and the Obama campaign have in common, then, is that they used their databases to tell stories. Or, more to the point, their databases and models were used so that Americans could tell stories to each other. Silver’s site became a virtual watercooler, especially towards the end of the campaign — a place where people would gather to talk about what was possible and what was likely. Nate’s voice helped to guide the discussions, but the real reason that he got such astonishing traffic was not that people wanted to read what he was writing, so much as that people were using his model as a framework within which to hold their own idiosyncratic discussions.

Similarly, the Obama campaign put enormous effort into making sure that when phone calls were made, the right people were talking to each other. Scripts are bad; one-to-one human connections are good. This is the age of Facebook, where big data meets the social graph: I’m sure the Romney campaign had a big database too, but it lacked the same storytelling ability, and lacked the same degree of insight into the possible connections that could be drawn within the universe of supporters and potential supporters.

Here’s another important part of the Scherer article:

For all the praise Obama’s team won in 2008 for its high-tech wizardry, its success masked a huge weakness: too many databases. Back then, volunteers making phone calls through the Obama website were working off lists that differed from the lists used by callers in the campaign office. Get-out-the-vote lists were never reconciled with fundraising lists. It was like the FBI and the CIA before 9/11: the two camps never shared data. “We analyzed very early that the problem in Democratic politics was you had databases all over the place,” said one of the officials. “None of them talked to each other.” So over the first 18 months, the campaign started over, creating a single massive system that could merge the information collected from pollsters, fundraisers, field workers and consumer databases as well as social-media and mobile contacts with the main Democratic voter files in the swing states.

This gave the Obama campaign a massive advantage over the Romney campaign. The Obama campaigns data was centralized and coordinated, while the Romney campaign relied in large part on SuperPACs which by law could not have access to the central database and could not be coordinated.

SuperPACs are dumb money. Their cash can almost never be effectively spent, because they’re not on the inside of the campaign. What’s more, because they’re not official campaigns, they always pay top dollar for their TV ad spots, rather than the discounted rates that stations are forced to offer to candidates. The Obama campaign determined, at various points, that if they approached potential donors with the message that Romney had a fundraising lead, that would help Obama raise more money for his own campaign. But the truth was that Romney’s fundraising lead was never particularly useful, because it wasn’t married to a coherent strategy and database. If anything, it just helped Obama raise more cash.

Obama is never going to run another campaign, so the advantage that Obama had over Romney does not necessarily mean that in the 2016 race the Democratic candidate is going to have a similar advantage. But we do know that the Democrats have the technology. And, at least for the time being, the Republicans don’t.


Comments are closed.