The downside of the beauty of physics

By Felix Salmon
November 3, 2011
excerpt of Emanuel Derman's new book today; I can highly recommend both the excerpt and the whole book.

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

We have an excerpt of Emanuel Derman’s new book today. I can highly recommend both the excerpt and the whole book, which is a very readable generalist’s guide — by a physicist-turned-quant — to models, their uses, and their abuses.

“First,” writes Derman, “one must recognize that there are no genuine theories in finance.”

To understand this, you need to understand how a physicist views a theory. So let me also excerpt for you one of the more wonderful passages in the book:

Newton’s theory is general and precise. The gravitational force is inversely proportional to exactly the square of the distance between the planets; Newton was confident that the power of the distance is precisely 2. Had he been a social scientist performing statistical regressions in psychology, economics, or finance, he would probably have proposed a power of 2.05 ± 0.31.

A theory is not a fetish; when it is successful (see the quantum theory of electricity and magnetism in chapter 5) it describes the object of its focus so accurately that the theory becomes virtually indistinguishable from the object itself. Maxwell’s equations are electricity and magnetism; the Dirac equation is the electron; the Weinberg-Salam model of weak and electromagnetic interactions matches the electrons and quarks in almost every detail, as closely as one can measure. You can layer metaphors on top of the equation, but the equation is the essence.

This takes me back to Nigel Wood, my great high-school physics teacher, and the three-volume Feynman Lectures on Physics that my dad gave me as a birthday present one year. It’s infectious and addictive stuff — but it’s also very dangerous when, as happens with great frequency, physicists come pale and blinking out of their post-doctoral studies and get thrown into the bowels of an investment bank somewhere.

The point is that when you’ve spent a decade or so in a world of true theories, it’s incredibly hard to understand, on a deep level, any series of pro-forma warnings about how there aren’t really any theories in finance and how models can explode. And it’s not just quants, it’s regulators, too, who pine for a world of certainty and who can be astonishingly blind to real-world risks that live outside their models.

How do we fix this problem? With great difficulty. Wilmott’s Certificate in Quantitative Finance is a start: if we’re going to have quants, and we are, then they should be trained well, in the real world. But I think in general this is one of those endemic and unhedgeable risks. And a decidedly unexpected side-effect of the amazing accuracy of forecasts in the world of quantum physics.

More From Felix Salmon
Post Felix
The Piketty pessimist
The most expensive lottery ticket in the world
The problems of HFT, Joe Stiglitz edition
Private equity math, Nuveen edition
Five explanations for Greece’s bond yield
Comments
16 comments so far

Not everyone has 15k to spend on Wilmott’s certificate.

Posted by jmh530 | Report as abusive

What? You mean the Black-Scholes equation is not virtually indistinguishable from an equity option itself? Damn, back to work.

Posted by EpicureanDeal | Report as abusive

I think this depends quite a bit on which branch of physics we’re talking about. Certain types of physicists and other physics-y scientists (people working on complex systems, mathematical biologists, climatologists, etc) are quite well acquainted with the phenomenon of data failing to match theory. Maybe the problem is that Wall Street is hiring too many theorists, and not enough experimentalists?

Posted by DrewSteen | Report as abusive

Maybe we need more business education in our schools. Not the stuff taught for business degrees, but the idea that your customers, suppliers, competitors, and regulators are out to screw you (or at least get your stuff for free). Artificial systems are going to get gamed by the people who use them, and it’s nearly impossible to model the various ways they will exploit the holes in your model or system.

Of course, when it’s in the best interests of all concerned to ignore the potential for cheating, the models will be given the status of truth.

Posted by AngryInCali | Report as abusive

At least with the recent financial disaster with the Gaussian Copula and say Long Term Capital Management, there were people who clearly understood the limits of the models — for instance, didn’t Merton or Scholes have several articles on the non-normality of errors and that “low probability events” actually were more likely than predicted by many models? — but ignored them for their own benefit.

Posted by InvisHand | Report as abusive

One can acknowledge the limits of one’s understanding and still not be able to manage events outside that understanding. This becomes far more likely when leveraging 30x.

Why is this even permitted? Even if the government doesn’t care, why would any individual want to risk their money that way?

Posted by TFF | Report as abusive

My understanding is that the guy responsible for the copula itself tried to warn people about the correlation problem, but by the time he did so, people were making buckets of money in markets that depended on using the copula to “prove” that there was no serious risk; they didn’t want to hear that they were actually gathering nickels in front of a steamroller.

Posted by Auros | Report as abusive

I think that you should also keep in mind that physics is basically the study of physical phenomena that can be represented accurately with differential equations. In other words when systems get too complicated to model well they stop being physics — when quantum physics becomes too complicated because there are too many types of atoms present it becomes chemistry, and when Newtonian physics gets complicated due to lots of interlocking components it becomes engineering. So I think that, as a generalisation, physicists are great solving complicated sets of equations but are ill-prepared for situations which are too complicated or uncertain to model accurately.

In my (engineering) PhD on particulate motion I noticed that physicists working in the same area would design their experiments in such a way as to try and get the particles to behave like a fluid so they could write down some tractable equations to describe their results. And when I was working for a quant-focussed equity manager in 2008 there was a sharp divide between people with an physics/mathematics background who wanted to keep the quant models running as normal and those from a trading/engineering background who thought that the new market dynamics made the old models invalid. (Economists were split: model guys fell in the first camp and data guys fell in the second.) At its heart was a difference of opinion as to whether the models were reflections of fundamental truths or just the best simplifications we could get of an unknowable system.

Posted by RowanS | Report as abusive

On the other hand, it’s often very hard to get physicists to understand that financial data is observational data, not experimental data and that displaying a neat power law won’t cut it for the rest of us who have already been thinking of this stuff before.

Posted by Th.M | Report as abusive

Note also that irrational risk-taking is a rational response to our money management incentives.

If I double your money, I get to keep 20%+ of the gains. If I lose your money, you get to keep 100% of the losses.

Best way to earn a fortune is to engage in schemes that will PROBABLY pay off. An occasional big loss isn’t good for the reputation of the traders involved, but their clients stand to lose more than they do.

Posted by TFF | Report as abusive

My impression is that the quants largely understood the limits of their models, but their bosses and the traders didn’t. The problem, then, isn’t that the quants came from a world that is more precise than the world of finance, but that they came from a world where they didn’t have to talk to anyone who didn’t think in models.

Posted by dWj | Report as abusive

“Even if the government doesn’t care, why would any individual want to risk their money that way?”

Because mostly, they’re risking other people’s money.

And while I say this all the time, the government doesn’t care about things like this, they only care about political problems. The losses that people absorb don’t seem to cause as many political problems as the possibility of eliminating the rights of other people to risk money that isn’t their own.

Posted by KenG_CA | Report as abusive

Derman’s “there are no genuine theories in finance” charge is, epistemologically, rather childish. Lots of silliness is avoided by just carefully defining terms.

Posted by GlibFighter | Report as abusive

The models were an excuse for risk hiding. The quickest way to make money in Wall Street is to offer risky investments, which are cheap to buy, and pass them off as safe investments. Basically, you earn the risk premium without assuming any of the risk. It often helps to have a magic selling phrase e.g. gold, high tech, internet, plastics, government approved, wonder drugs, or more recently, scientifically hedged by physics PhD types.

The whole point of hiring “quants” and all that complex jiggery pokery was no different than the hand movements of a three card monte jack – distraction. That there were so many suckers speaks ill of our many financial institutions but well of PT Barnum who had this pegged.

Posted by Kaleberg | Report as abusive

“The point is that when you’ve spent a decade or so in a world of true theories, it’s incredibly hard to understand, on a deep level, any series of pro-forma warnings about how there aren’t really any theories in finance and how models can explode.”

I think you’ve got this backwards. Anyone who’s spent a decade or so in physics without confronting an array of exploding models must have been working on something pretty boring. Quantum Mechanics is amazingly accurate but physicists don’t sit around testing the same thing over and over again.

When banks find/found it is easier to sell crap with the blessing of a Phd, they find one to do so; that’s not an indictment of the scientific method.

Posted by seanl | Report as abusive

The people who know about the lack of certainty and behave like a professional gambler playing the opponent and not the bank are winning (George Soros).

The people who think they can model reality and believe in those models go bust (LTCM).

Posted by Finster | Report as abusive
Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/