Barack Obama and the limitations of probabilistic decision making

By Felix Salmon
September 5, 2012
Michael Lewis has a big profile of Barack Obama in the latest Vanity Fair, and Obama tells Lewis something very interesting:

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

Michael Lewis has a big profile of Barack Obama in the latest Vanity Fair, and Obama tells Lewis something very interesting:

“Nothing comes to my desk that is perfectly solvable,” Obama tells Lewis. “Otherwise, someone else would have solved it. So you wind up dealing with probabilities. Any given decision you make you’ll wind up with a 30 to 40 percent chance that it isn’t going to work. You have to own that and feel comfortable with the way you made the decision. You can’t be paralyzed by the fact that it might not work out.”

This is very much in line with the m.o. of Larry Summers, and of Bob Rubin before him. Here’s how Steve Rattner explains it, in his book about the auto bailouts:

Larry pressed us to attach probabilities to our recommendations and countered with odds of his own. Like Bob Rubin, with whom the concept is most closely associated, Larry is an enthusiast for “probabilistic decision making,” a method for weighing uncertainties.

This all sounds very scientific — but the problem is, it isn’t. Certainly, if you’re making a decision like whether or not to surge in Afghanistan, you cannot know for sure what the consequences of that decision are going to be. But trying to express things in probabilistic terms is not much of an improvement.

For one thing, there’s no evidence to believe that people are better at subjectively assigning probabilities to outcomes than they are at simply predicting what’s going to happen. People in general — and experts like Summers in particular — tend to suffer from overconfidence bias, and tend to massively overestimate the probability of things they think are going to happen. And if you get too deep into the weeds of probabilistic decision making, you end up multiplying probabilities, and thereby massively compounding your estimation errors.

On top of that, probabilistic decision making tends to live in a binary world: what is the chance of success, and what is the chance of failure? But that kind of analysis can conceal much more than it reveals, and nearly always results in people doing things which have a very high probability of success, no matter how devastating failure would be. This is what I call the Rubin Trade, and is very popular among merger arbs, which is the business Rubin started in. Let’s say that company X has agreed to buy company Y for $100 per share, but if the merger falls through, then Y’s stock will fall to $50. Would you buy stock in Y for $95? People who do that — especially if they leverage themselves in doing so — will normally make a lot of money, and cash large bonuses. But once in a while the trade will blow up. In many ways, the entire financial crisis was the result of a huge global bet that very safe securities were very safe.

If you’re doing probabilistic analysis, then, you really need to be looking at probability distributions, rather than simply drawing a line between success and failure and trying to work out the ratio of the areas on the left and right sides of that line. Specifically, you need to be looking at tail risk, and asking yourself just how bad things could get if they do go wrong. And the flipside of that is that sometimes it’s a good idea to bet on things which are unlikely to happen, just because the payoff should those things happen is so big. That’s basically what a large part of options trading is all about: trying to buy things which are cheap and which will probably expire at zero, but which if they don’t can pay off enormously.

Summers understands all this. He’s one of the more forceful proponents saying that it can be a good idea to deliberately go too far on the side of doing too much, because the risks of doing too little are so great. But it’s hard to build that kind of analysis into probabilistic decision making — unless you’re the person framing the questions. And that’s the real problem with this kind of framework: you can pretty much always get any answer you want, just by being careful about the questions you’re asking, and how you use the answers you get. Let’s go back to Rattner:

At one point, [Summers] confessed that as we gave our answers, he was discounting our probabilities based on what he thought we would say. For example, knowing that Ron was in favor of saving Chrysler, Larry lowered the probability Ron assigned to the success of the alliance with Fiat. The opposite for Harry. Plainly, Larry was loving this debate…

Larry called for a show of hands. His question was precise: “If you assume that the probability is 50 percent or greater that Chrysler would survive for five years, would you save it?”

Diana was unhappy with the phrasing, because she thought Larry was stacking the deck — forcing those who believed Chrysler’s chances were actually slim to assume a higher probability. She had suspected that he wanted to save Chrysler and now was sure of it. While she recognized that under Larry’s formulation she should be voting to save Chrysler, she voted against it anyway, as a kind of protest. Austan felt sandbagged too.

It’s pretty obvious, here, that the reason Larry was loving the debate was precisely because he was controlling it. If you get to unilaterally change other people’s probabilities, and you get to frame all the questions, you can basically give yourself dictator-like powers while ostensibly running a democratic, technocratic, and probabilistic process.

And this is why I worry about the way that Obama has internalized this way of thinking: when he thinks he’s making a choice, in fact he’s always going to end up choosing the option that his advisers have managed to present as having the highest probability of success. And after making those decisions, he won’t lose sleep about their possible downsides or unintended consequences.

For instance, consider the decision to concentrate on healthcare rather than climate change as the Obama administration’s first big legislative push. In the wake of that decision, the chances of getting any kind of climate-change legislation passed became effectively zero — and the negative consequences for the well-being of the planet as a whole could easily end up dwarfing the upside from everything else the Obama team does put together.

Meanwhile, no one really worried, when the healthcare legislation was being negotiated, that the whole thing could end up being struck down by the Supreme Court. It wasn’t, in the end — but it turned out to be very close. When calculating healthcare probabilities, did anybody in the White House properly account for SCOTUS risk? Almost certainly not. And as a result the whole calculation was missing a crucial element.

For Obama, if a decision doesn’t work out, he’s OK with that, since he reckons there’s a statistical certainty that a good third of his decisions will fail to work out. And he takes solace in the idea that so long as the process of arriving at the decision was a good one, by which he means that it was properly technocratic and probabilistic, then he did the best that he could have done.

But that kind of decision-making framework leaves very little room for ideals — for actually putting into practice the kind of vision you have for America. By making decisions on a case-by-case basis, you can end up missing out on building something bigger and much more coherent. In 2008, America voted for a man who was truly excellent at staring into the distance, a man looking at the big picture, and at a centuries-long legacy. Instead, hampered by the financial crisis and by a dysfunctional Congress, they got a man who spends his days weighing success probabilities: a tactician, rather than a strategist.

I’m sure that when Obama accepts his party’s nomination for president in Charlotte on Thursday, there will be lots of moving and rousing rhetoric. But realistically, we cannot hope that a second-term Obama will have much if any ability to make his high-flying dreams a reality. Especially not if he continues to make decisions based on which actions have a 65% probability of success.

29 comments

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

Agree with most of this. I hate sharing political views (speaking of bias), but I think Obama has been largely hampered by limitations he perceives in himself, not by Congress or the financial crisis. On the other hand, Romney may well be hampered by the lack of limitations he perceives in himself. Too many of our elections seem to be framed in similar ways.

Posted by Curmudgeon | Report as abusive

You’ve misinterpreted the comment in the Michael Lewis story. At most, the article suggests that Obama realizes that every decision has a significant chance of going wrong. Your basic criticism is that probabilistic decision making doesn’t allow gambles on low-probability, high-reward choices. That may be right, but nothing in the article suggests that Obama adheres to that kind of decision making. What about Obama’s decision to double-down with additional troops in Afghanistan?

Posted by jacoxnet | Report as abusive

This is a seriously overstretched analysis, at least so far as Obama goes. All the guy is saying is that you can’t get frozen making critical decisions by the reality that they might not work out or they might be wrong. All you can do is your moral best and accept the limits of being a mere fallible human being. I can’t say what Summers is about when (or if) he tries to spin debates with verbal and quasi-logical tricks. But there really is nothing to connect Obama’s frank remarks (which pretty much remind me of the spirit of Rumsfeld’s famous remarks about “known and unknown unknowns”) with the picture painted here of Summers. This is just milking an honest sentence for an opinion piece. Try again when you’re back from your mental summer vacation.

Posted by From_California | Report as abusive

Not long ago we had a president who believed, without justification, that his decisions would have a 100% chance of success. The results were closer to 0%.

While I agree that making decisions on a case-by-case basis is not the best way to manage any large entity, that’s all that our political system will allow. It’s even worse, as those decisions are always (not usually, but always) subject to deal making, instead of compromise. Every decision is made in a silo, as if it has no impact on the rest of the nation. The deal making means that the presiden accepts a bill that he believes is bad from Congress, in exchange for Congress passing what he feels is even more important. Those decisions are not made to solve problems other than political ones. A compromise is not the same thing; it usually affects the magnitude of the intended solution.

Curmudgeon, I have to disagree with you here. I can’t see how you can ignore the fact that almost everything Obama could propose during his term would be filibustered, no matter how far to the right his idea might be. He started with handcuffs, and he would have to be blind not to see the limitations they would put on him.

Reuters, please fix your commenting system. It isn’t working well at all, and since you don’t seem to be monetizing it, you should use something like disqus or livefyre.

Posted by KenG_CA | Report as abusive

Felix, I’m not sure there’s sufficient evidence/transparency about his process to state how he factors in tail risks (or even expected values of his decisions). Whether the President left out this level of nuance or Lewis abridged it is hard to say, but this doesn’t necessarily imply that this excerpt accurately reflects the President’s decision-making process. I wouldn’t, for example, leap to assume that SCOTUS risk was left out of discussions of health care legislation, although perhaps you know something I don’t.

Your point that his presidency has been tactical rather than strategic certainly feels right, and in a way sums up much of the criticisms he’s taken from his party. And I don’t think there’s any question that in this sort of decision-making process he who wins right to frame the debate in the oval office can largely predetermine the outcome. This is a problem that’s very hard to evade, no matter the decision process of a given leader — I think of Cheney’s use of unvetted CIA info to sway President Bush’s foreign policy decisions, and he was anything but a technocrat. The trick, as always, is to intellectually honest advisers — easier said than done.

Posted by dmcdougall | Report as abusive

I think Felix you are confusing too many things here. The choices for which Obama is assigning probabilities are all already ‘ideologically compatible’ choices. You are completely discounting that. Say to save Auto-industry; to save it and to save corresponding jobs as much as possible; that was the basic ideological policy decision. Romney is on record saying that he would not have done that and you have Bloomberg Editorial arguing how ‘saving Manufacturing jobs may be a good politics; but not a good policy’. That is the ideological / political argument. Your faulty analysis implies that Obama does not have any ‘ideology or political belief system’. This is a horribly out of reality statement – GOP slams Obama for being too ideological and Left wants him more. But to claim he does not have any belief system and he just sits in the Oval Office ‘popping up’ probability numbers; is a totally careless journalism.

How is that no one edited what you are writing here?

Obama has faults and he has ducked leadership issues too. Those are all valid political criticism and clearly he has to answer those. You may or may not vote him, but portraying him as the ‘clueless person’ as like in Allen Turing’s machine assigning random probability numbers to choices and then picking up a choice; that is way too much….

Posted by umeshgeeta | Report as abusive

If the only choice Obama faced was choosing between healch care and climate change, then health care was a no brainer. “Climate change” is simply a euphemism for increasing the cost of energy, which would have turned a bad recession into an actual depression. The kicker is, nothing the U.S. could possibly do to reduce emissions could possibly offset the enormous increases in emissions being generated by India and China.

With regard to health care, Obama, like Clinton before him, fell prey to the classic planner’s fantasy that you have to do everything at once. He should have taken a leaf from FDR’s book: the original Social Security Act deliberately avoided serving the poor. They were too controversial. Obama should have found some way to make people “earn” their health coverage. Yes, that would have been cold, but not nearly as cold as killer drones.

Posted by AlanVanneman | Report as abusive

If the only choice Obama faced was choosing between healch care and climate change, then health care was a no brainer. “Climate change” is simply a euphemism for increasing the cost of energy, which would have turned a bad recession into an actual depression. The kicker is, nothing the U.S. could possibly do to reduce emissions could possibly offset the enormous increases in emissions being generated by India and China.

With regard to health care, Obama, like Clinton before him, fell prey to the classic planner’s fantasy that you have to do everything at once. He should have taken a leaf from FDR’s book: the original Social Security Act deliberately avoided serving the poor. They were too controversial. Obama should have found some way to make people “earn” their health coverage. Yes, that would have been cold, but not nearly as cold as killer drones.

Posted by AlanVanneman | Report as abusive

KenG, I might almost agree with you, except that Obama had both houses of Congress, including a supermajority in the Senate, for two full years, and could only just barely pass health care. People tend to forget that. That isn’t a good sign for a sitting President.

Posted by Curmudgeon | Report as abusive

“In many ways, the entire financial crisis was the result of a huge global bet that very safe securities were very safe.”

I understand the snark in the phrase “very safe securities” but this isn’t really true. The financial crisis was the result of grifters and their enablers packaging securities that they knew to a moral certainty were not safe and passing them off. The financial markets blew up because of fraud, not because some bizarre long-tail noboddycuddanoed risk came to pass.

Similarly, 9-11 was not the result of an unprecedented nobody-could-have-foreseen event. It was the result of an administration that didn’t care about terrorism and couldn’t be bothered to pay attention to people who knew what they were talking about.

Over and over again, we are told that fraud, negligence, self-dealing, and simple stupidity are not what they seem to be but are the result of extraordinarily improbable events. No one is ever at fault – not for losing a billion dollars of client money (MF Global), not for drowning a city (Katrina), not for endless unwinnable war (Afghanistan). We’re plagued by an endlessly flying flock of black swans.

Posted by Blox | Report as abusive

As a result of the decisions made by the Politburo Standing Committee of the Communist Party of China, the negative consequences for the well-being of the planet as a whole WILL easily end up dwarfing the upside from everything the Obama team does put together, and that would have been true even if the Obama Administration decided to concentrate on climate change rather than healthcare as its first big legislative push.

Posted by Strych09 | Report as abusive

The commenter who thinks that Obama hurts himself more than Congressional obstructionism or Bush-era budget constraints is living in a dream world.

No president in living memory has ever met such a solid wall of stupidity like this Republican House, acid disrespect from plutocrat shills on the Supreme Court or budgetary harakiri from his predecessor (“Starve the Beast” anyone?). Not to mention the constant u r i n a t i o n in the well of public discourse from Fox News and talk radio.

In fact, you have to wonder how important the question of Obama’s decision-making is, given the implacable antagonism he gets from his political foes.

Posted by Panskeptic | Report as abusive

Given the implacable obstructionism of Obama’s opponents, in the Congress and on the Supreme Court, plus the poisonous drumbeat of disinformation from Fox News and talk radio, I don’t really see analyzing Obama’s decision-making process as a terribly urgent subject.
A more fruitful question might be, how does he get anything implemented?

Posted by Panskeptic | Report as abusive

“For instance … the chances of getting any kind of climate-change legislation passed became effectively zero — and the negative consequences for the well-being of the planet as a whole could easily end up dwarfing the upside from everything else the Obama team does put together.”

Glad you brought it up, and here’s my question: given climate change’s importance, why don’t you write about it more? There are PLENTY of finance angles, even leaving aside the whole you-may-not-be-interested-in-{war/the-di alectic/climate-change}-but-it-is-intere sted-in-you angle.

You clearly here the call, so: please answer it.

Posted by SamPenrose | Report as abusive

“For instance … the chances of getting any kind of climate-change legislation passed became effectively zero — and the negative consequences for the well-being of the planet as a whole could easily end up dwarfing the upside from everything else the Obama team does put together.”

Glad you brought it up, and here’s my question: given climate change’s importance, why don’t you write about it more? There are PLENTY of finance angles, even leaving aside the whole you-may-not-be-interested-in-{war/the-di alectic/climate-change}-but-it-is-intere sted-in-you angle.

You clearly here the call, so: please answer it.

Posted by SamPenrose | Report as abusive

This is bullshit. You are misconstruing the quote. Obama was merely stating that he realizes that every decision has a risk of success and of failure. This is an important concept as it would be foolish to view events as predetermined. If we look back on a decision solely on whether or not it was correct and ignore the decision making process than we don’t really learn anything, as we ignore all the possibilities that were not realized. Obama is obviously not basing his decisions on some ridiculous mathematical system, rather a rational decision making process that accounts for multiple outcomes.

Posted by nbrown14 | Report as abusive

Felix, I’ve criticized positions you have taken in the past, but this piece makes perfect sense. When you observed that the process used “…all sounds very scientific — but the problem is, it isn’t”, you nailed it.

In stating that “If you get to unilaterally change other people’s probabilities, and you get to frame all the questions, you can basically give yourself dictator-like powers while ostensibly running a democratic, technocratic, and probabilistic process”, you ring the bell again.

When you conclude that “…realistically, we cannot hope that a second-term Obama will have much if any ability to make his high-flying dreams a reality”, you answer the primary question of the coming election. That is “Why should I cast a vote FOR or AGAINST a second Obama term?

Posted by OneOfTheSheep | Report as abusive

Good job, Felix.
Decades ago, Kahneman & Tversky showed that even experienced professionals suffer from inherent flaws in the way they judge and evaluate data, problems, risks, and outcomes.
It goes to the fact that we’re all just primates who not long ago lived up treetops and in caves, and our basic cognitive capabilities are mainly the product of those natural environments.
So next time you think central bankers, analysts, CEOs or politicians know what they’re talking about, let alone what they’re doing – think again…

Posted by reality-again | Report as abusive

Felix, I’ve criticized positions you have taken in the past, but this piece makes perfect sense. When you observed that the process used “…all sounds very scientific — but the problem is, it isn’t”, you nailed it.

In stating that “If you get to unilaterally change other people’s probabilities, and you get to frame all the questions, you can basically give yourself dictator-like powers while ostensibly running a democratic, technocratic, and probabilistic process”, you ring the bell again.

When you conclude that “…realistically, we cannot hope that a second-term Obama will have much if any ability to make his high-flying dreams a reality”, you answer the primary question of the coming election. That is “Why should I cast a vote FOR or AGAINST a second Obama term?

Posted by OneOfTheSheep | Report as abusive

Love this piece – vintage FS it is; cryin’ shame the 6-of-11 comments I can see don’t – as a group – quite rise to the level of the post; not really surprising though, is it?

Bob Rubin earned his spurs as an ‘arb’, a trade where hundreds of individual transactions make ‘probability theory’ exquisitely applicable, much as it is in a biz like life insurance. (‘Inside info’ works even better – Rubin’s GS crew was good at that too.) Larry Summers never earned his spurs as anything other than an academic/bureaucrat – gotta wonder if he ever got the drift of what Rubin was talking about. Obama has no spurs; not an exaggeration to say The Presidency is the first full-time gig he’s had in his entire life. He’s in so far over his head that light barely penetrates to that depth – why expect anything from him at all, except self-promotion? He’s good at that.

Playing the odds makes sense on The Street and at the track or at the dice table, where risk can be controlled by limiting the wager, and be distributed over many chance-events. In ‘one-off’ public policy matters, something different is required. Rubin probably gets that; who knows about Summers? Obama ….

Posted by MrRFox | Report as abusive

Love this piece – vintage FS it is. The few comments visible don’t quite do justice to the OP.

Bob Rubin earned his spurs as an ‘arb’, a line of trade where hundreds of individual transactions make ‘probability theory’ exquisitely applicable, much as it is in a biz like life insurance. (‘Inside info’ works even better – Rubin’s GS protégé was feloniously good at that too.) Larry Summers never earned his spurs as anything other than an academic/bureaucrat – gotta wonder if he ever got the drift of what Rubin was talking about. Obama has no spurs; not an exaggeration to say The Presidency is the first full-time gig he’s had in his entire life. He’s in so far over his head that light barely penetrates to that depth – why expect anything from him at all, except self-promotion? He’s good at that.

Playing the odds makes sense on The Street and at the track or at the dice table, where risk can be controlled by limiting the wagers, and be distributed over many chance-events. In ‘one-off’ public policy matters, something different is required. Rubin probably gets that; who knows about Summers? Obama ….

Posted by MrRFox | Report as abusive

Life is not black and white, buddy. If it is to you, get some sunglasses and enjoy the day. As for Larry Summers, I think he got kicked out.

Posted by Tseko | Report as abusive

Curmudgeon, Obama never really had control of either house, both were limited by conservative Democrats who often split with their party. The Democrats had no Tom DeLay to keep everyone in line, and as a result, many (especially newly elected house members) forgot Obama brought them to the party, and abandoned him (they paid the price in 2010). Also, Edward Kennedy died in 2009, leaving the senate with only 59 democrats for most of that first two years. It takes 60 votes to stop a filibuster, and not only did they rarely get 60 votes, they couldn’t even get Democratic senators to step up and force the filibusters – they basically caved at the threat of a filibuster.

Neither Pelosi nor Reid had control of their chamber, and they were more to blame for the lack of legislative success than Obama. Also, when he could get 60 votes in the senate, he had to make deals with people on both sides, which totally distorted his original goals (I don’t believe the health care plan that passed was anywhere near what he originally wanted).

Posted by KenG_CA | Report as abusive

I’m pretty sure the White House was aware of the Constitutional authority risk that the individual mandate represents when they were crafting the ACA. Certainly in the winter of ’10, the internet had quite a few discussions on the topic prior to passage.
I think part of the limits of probability based decisions is that 99% of all decisions Obama is going to make, a large portion of the GOP will go crazy about. Independent of whether they said or supported something identical up to that very minute. It’s also difficult to assign probability when the referees (the media) have a bias to present both sides of every issue, even for those issues where one side is fraudulent and/or inane.

Posted by thispaceforsale | Report as abusive

KenG…he had 2 years that were filibuster free. He didnt start with handcuffs, he started with clear blue water, both the senate and the house…and it was all up to him to manage. And he didnt.

Posted by Dietzamerk | Report as abusive

KenG…he had 2 years that were filibuster free. He didnt start with handcuffs, he started with clear blue water, both the senate and the house…and it was all up to him to manage. And he didnt.

Posted by Dietzamerk | Report as abusive

KenG…he had 2 years that were filibuster free. He didnt start with handcuffs, he started with clear blue water, both the senate and the house…and it was all up to him to manage. And he didnt.

Posted by Dietzamerk | Report as abusive

KenG…he had 2 years that were filibuster free. He didnt start with handcuffs, he started with clear blue water, both the senate and the house…and it was all up to him to manage. And he didnt.

Posted by Dietzamerk | Report as abusive

KenG…I agree with you on the commenting system…no clue whether my initial response will get posted 5 times or not at all, but my point was that Obama was in no way handcuffed when he started, he was filibuster-free for his first two years. How do you explain your comment that they put limitations on him?

Posted by Dietzamerk | Report as abusive