Opinion

Felix Salmon

When quants tell stories

Felix Salmon
Nov 7, 2012 22:11 UTC

The dominant narrative, the day after the presidential election, is the triumph of the quants. As Simon Jackman notes, essentially every single poll-averaging quant — Jackman himself, Drew Linzer, Sam Wang, you name it — managed to call every single state plus the presidential election: an astonishing 51/51 success rate.

That was part skill and part luck, as all such things are. Here’s the final electoral-vote probability distribution from Nate Silver, the most famous of the quants:

Obama ended up winning 332 electoral votes, assuming that he ends up winning Florida; that was the single most likely outcome in Silver’s model, with a probability of just over 20%. But the mode outcome isn’t the median outcome: the official Nate Silver forecast was that Obama would get 313 votes. Roughly 80% of the time, under Silver’s model, he would have ended up calling at least one state wrong. So Silver did get lucky.

Silver is the most visible of the quants, partly because of his perch at the NYT, partly because he has a new book out, and partly because he’s very good at taking his complex mathematical model and turning it into bite-sized English-language blog posts. If you think that the value of Nate Silver is in the model, you’re missing the most important part: there are lots of people with models, and most of those models are pretty similar to each other. The thing which sets Silver apart from the rest is that he can write: he can take a model and turn it into a narrative, walking his readers through to his conclusions.

Which brings me to Michael Scherer’s great story about the Obama-campaign quants: the people who A/B tested everything and who built an astonishingly formidable ground game. Rather than just blanketing the airwaves with ads they thought were good, the Obama campaign was constantly testing, quantifying, and targeting.

At heart, the campaign was marrying quantitative skills with storytelling, to unbeatable effect. Which stories should the campaign tell, to any given group of people? How should it tell those stories? And who should it get to deliver those stories? The database answered all those questions:

Cash raised online came through an intricate, metric-driven e-mail campaign in which dozens of fundraising appeals went out each day. Here again, data collection and analysis were paramount. Many of the e-mails sent to supporters were just tests, with different subject lines, senders and messages. Inside the campaign, there were office pools on which combination would raise the most money, and often the pools got it wrong…

In the final weeks of the campaign, people who had downloaded an app were sent messages with pictures of their friends in swing states. They were told to click a button to automatically urge those targeted voters to take certain actions, such as registering to vote, voting early or getting to the polls. The campaign found that roughly 1 in 5 people contacted by a Facebook pal acted on the request, in large part because the message came from someone they knew…

“We were able to put our target voters through some really complicated modeling, to say, O.K., if Miami-Dade women under 35 are the targets, [here is] how to reach them,” said one official. As a result, the campaign bought ads to air during unconventional programming, like Sons of Anarchy, The Walking Dead and Don’t Trust the B—- in Apt. 23, skirting the traditional route of buying ads next to local news programming. How much more efficient was the Obama campaign of 2012 than 2008 at ad buying? Chicago has a number for that: “On TV we were able to buy 14% more efficiently … to make sure we were talking to our persuadable voters,” the same official said.

The thing that Silver and the Obama campaign have in common, then, is that they used their databases to tell stories. Or, more to the point, their databases and models were used so that Americans could tell stories to each other. Silver’s site became a virtual watercooler, especially towards the end of the campaign — a place where people would gather to talk about what was possible and what was likely. Nate’s voice helped to guide the discussions, but the real reason that he got such astonishing traffic was not that people wanted to read what he was writing, so much as that people were using his model as a framework within which to hold their own idiosyncratic discussions.

Similarly, the Obama campaign put enormous effort into making sure that when phone calls were made, the right people were talking to each other. Scripts are bad; one-to-one human connections are good. This is the age of Facebook, where big data meets the social graph: I’m sure the Romney campaign had a big database too, but it lacked the same storytelling ability, and lacked the same degree of insight into the possible connections that could be drawn within the universe of supporters and potential supporters.

Here’s another important part of the Scherer article:

For all the praise Obama’s team won in 2008 for its high-tech wizardry, its success masked a huge weakness: too many databases. Back then, volunteers making phone calls through the Obama website were working off lists that differed from the lists used by callers in the campaign office. Get-out-the-vote lists were never reconciled with fundraising lists. It was like the FBI and the CIA before 9/11: the two camps never shared data. “We analyzed very early that the problem in Democratic politics was you had databases all over the place,” said one of the officials. “None of them talked to each other.” So over the first 18 months, the campaign started over, creating a single massive system that could merge the information collected from pollsters, fundraisers, field workers and consumer databases as well as social-media and mobile contacts with the main Democratic voter files in the swing states.

This gave the Obama campaign a massive advantage over the Romney campaign. The Obama campaigns data was centralized and coordinated, while the Romney campaign relied in large part on SuperPACs which by law could not have access to the central database and could not be coordinated.

SuperPACs are dumb money. Their cash can almost never be effectively spent, because they’re not on the inside of the campaign. What’s more, because they’re not official campaigns, they always pay top dollar for their TV ad spots, rather than the discounted rates that stations are forced to offer to candidates. The Obama campaign determined, at various points, that if they approached potential donors with the message that Romney had a fundraising lead, that would help Obama raise more money for his own campaign. But the truth was that Romney’s fundraising lead was never particularly useful, because it wasn’t married to a coherent strategy and database. If anything, it just helped Obama raise more cash.

Obama is never going to run another campaign, so the advantage that Obama had over Romney does not necessarily mean that in the 2016 race the Democratic candidate is going to have a similar advantage. But we do know that the Democrats have the technology. And, at least for the time being, the Republicans don’t.

COMMENT

@Woltmann, the whole concept of a national poll is flawed for exactly that reason. Anybody who bothers to conduct a national poll is clearly not interested in who will win the election.

Posted by TFF | Report as abusive

The downside of the beauty of physics

Felix Salmon
Nov 3, 2011 19:24 UTC

We have an excerpt of Emanuel Derman’s new book today. I can highly recommend both the excerpt and the whole book, which is a very readable generalist’s guide — by a physicist-turned-quant — to models, their uses, and their abuses.

“First,” writes Derman, “one must recognize that there are no genuine theories in finance.”

To understand this, you need to understand how a physicist views a theory. So let me also excerpt for you one of the more wonderful passages in the book:

Newton’s theory is general and precise. The gravitational force is inversely proportional to exactly the square of the distance between the planets; Newton was confident that the power of the distance is precisely 2. Had he been a social scientist performing statistical regressions in psychology, economics, or finance, he would probably have proposed a power of 2.05 ± 0.31.

A theory is not a fetish; when it is successful (see the quantum theory of electricity and magnetism in chapter 5) it describes the object of its focus so accurately that the theory becomes virtually indistinguishable from the object itself. Maxwell’s equations are electricity and magnetism; the Dirac equation is the electron; the Weinberg-Salam model of weak and electromagnetic interactions matches the electrons and quarks in almost every detail, as closely as one can measure. You can layer metaphors on top of the equation, but the equation is the essence.

This takes me back to Nigel Wood, my great high-school physics teacher, and the three-volume Feynman Lectures on Physics that my dad gave me as a birthday present one year. It’s infectious and addictive stuff — but it’s also very dangerous when, as happens with great frequency, physicists come pale and blinking out of their post-doctoral studies and get thrown into the bowels of an investment bank somewhere.

The point is that when you’ve spent a decade or so in a world of true theories, it’s incredibly hard to understand, on a deep level, any series of pro-forma warnings about how there aren’t really any theories in finance and how models can explode. And it’s not just quants, it’s regulators, too, who pine for a world of certainty and who can be astonishingly blind to real-world risks that live outside their models.

How do we fix this problem? With great difficulty. Wilmott’s Certificate in Quantitative Finance is a start: if we’re going to have quants, and we are, then they should be trained well, in the real world. But I think in general this is one of those endemic and unhedgeable risks. And a decidedly unexpected side-effect of the amazing accuracy of forecasts in the world of quantum physics.

COMMENT

The people who know about the lack of certainty and behave like a professional gambler playing the opponent and not the bank are winning (George Soros).

The people who think they can model reality and believe in those models go bust (LTCM).

Posted by Finster | Report as abusive
  •