How macroeconomic statistics failed the US

By Felix Salmon
September 27, 2011

There’s one big reason why the current economic weakness in the US has come as such a shock. It’s not the only reason, but it’s an important one, and it hasn’t gotten nearly the attention it deserves: the state of macroeconomic data-gathering in the US is pretty weak.

In particular, the data coming out of the Bureau of Economic Analysis at the beginning of 2009 was way off. Here’s Cardiff Garcia, introducing an interview with Fed economist Jeremy Nalewaik:

The initial GDP estimate for the fourth quarter of 2008 showed that the economy contracted by 3.8 per cent. It was released on January 30, 2009 — about three weeks before Obama’s first stimulus bill passed. That number was continually adjust down in later revisions, and in July of this year the BEA revised it all the way down to a contraction of 8.9 per cent.

The BEA is happy to try to explain what happened here — but whatever the explanation, the original 3.8% figure was a massive and extremely expensive fail. It was bad enough to be able to get a $700 billion stimulus plan through Congress, but if Congress and the Obama Administration had known the gruesome truth — that the economy was contracting at a rate of well over $1 trillion per year — then more could and would have been done, both at the time and over subsequent months and years. Larry Summers warned at the time that the risks of doing too little were much greater than the risks of doing too much; only now do we know just how right he was on that front. (And even he didn’t push for a stimulus of more than $700 billion.)

So what’s being done to beef up the state of America’s macroeconomic statistics so that this kind of monster error doesn’t happen again? The BEA is doing the best it can, but it’s constrained both in terms of its budget and in terms of the quality of economists it can attract.

Here’s how Cardiff ended his interview:

FT Alphaville was recently having a broader discussion about the status of macroeconomic data-gathering in the US with a fellow blogger, Felix Salmon, and he made the point to us that it’s been in secular decline for the last few decades. Do you agree? Is this something that’s come up in your work on output measures?

Some evidence suggests that the measurement errors in GDP growth have become worse in recent years. This may have to do with the increasing importance of services in the economy in recent decades, a sector where the GDP source data has historically been spotty. This is because, historically, the U.S. Census bureau has not collected spending data for many types of services on a regular basis.

Despite budget constraints, the statistical agencies have mounted a major effort to improve their measurement of services GDP. However, even as they make progress, it is important to keep in mind that there will always be measurement errors of some kind or another in the GDP and GDI source data, so taking some sort of weighted average, as I proposed in the Brookings paper, would be the soundest approach.

Frankly, this just isn’t good enough. Moving to a weighted average of GDP and GDI doesn’t improve the quality of our statistics one bit; it’s just an attempt to cope with the fact that neither of them is particularly reliable. As the economy becomes increasingly complex and service-based rather than goods-based, it’s crucial that our statistical architecture keeps pace — and it clearly isn’t doing so.

When I told Cardiff that the status of macroeconomic data-gathering has been declining for decades, I was making two separate statements — first that the quality of statistics has been declining, and secondly that the status of economists collating such statistics has been declining as well. Once upon a time, extremely well-regarded statisticians put lots of effort into building a system which could measure the economy in real time. Today, I can tell you exactly how many hot young economists dream of working for the BEA on tweaks to the GDP-measurement apparatus: zero.

I’m pessimistic that this is going to change. Putting together macroeconomic statistics is not a prestigious part of the economics profession any more, and government payscales are pretty meager compared to what good economists earn elsewhere.

Increasingly the economists in the government who craft the policy responses to macroeconomic developments are working on a GIGO (garbage in, garbage out) basis. That, in turn, means more bad responses, more bubbles, more recessions, and in general more macroeconomic volatility. The world is getting messier — and we don’t even have a good basis for measuring just how messy it is, any more.

14 comments

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

Your critique is misguided. Props for linking to the BEA’s explanation, but why dismiss it out of hand? “Advance” estimates always have a significant margin of error, in large part because they are based on data from only 2/3 of the quarter. So December 2008 was worse than October and November. They got much more accurate a month later with the preliminary numbers (-6.2%).

Every BEA release includes a note at the end looking at their historical forecast accuracy, and I’d encourage you to look at them. They’ve computed standard deviations of the estimate error for each time period, and the advance releases get real GDP off by a full 1.0% from advance to the latest estimate.

Yes, this error was significantly bigger than the historical average (don’t call it an “n-sigma” event unless you’re feeling obnoxious). And yes, working at the BEA doesn’t sound like much fun.

But since a big part of economic forecasting requires the use of backward-looking smoothing techniques (do you have a better method in mind?), estimates are necessarily going to be more inaccurate when you have very large, unprecedented discontinuities in the time series.

Rail about black swans and the inadequacies of statistics if you like, but the problem here isn’t the messenger. Looking at the numbers, I’m convinced that the BEA puts out the best possible estimate of GDP at any given moment in time.

Posted by loudnotes | Report as abusive

Even so, a downgrade from 3.9% to 8.9% is a pretty big error – most other countries make adjustments in the are after the decimal point, not after it. Of course, one can never rule out political interference, or at the very least, political sympathy feeding in favourable figures to cover up the damage that a spend of 48% of GDP per year on the military can do to an economy.

To put that in perspective, the EU are in total with a larger economy than the US spends just 20% of GDP on defence. That’s a saving of $500 billion a year. So, if the EU area needs a two trillion Euro rescue fund, how much bigger will the US pot need to be to cover all that waste going into the US defense budget?

The US is spending too much on defense and it isn’t sustainable. The money saved could make a big difference to the economy, and easily pay for a better measurement system.

Posted by FifthDecade | Report as abusive

“a spend of 48% of GDP per year on the military”

FifthDecade, I get the feeling that you and I are working from different definitions of “GDP”.

The US doesn’t spend anywhere close to 48% of its gross domestic product on the military, no matter how you define it.

Posted by TFF | Report as abusive

Sorry, I meant 48% of all military spending worldwide.
http://www.visualeconomics.com/military- spending-worldwide/

Posted by FifthDecade | Report as abusive

How can my credit card purchase data be used as a macro statistic and how can I benefit from this data?

It seems like I’m generating all this useful data naturally and it just sits in a vault. I’d be plenty happy to give up my privacy for the right price!

Posted by djiddish98 | Report as abusive

Not to blogwhore, but given what people, ahem, learn from macro data (http://www.angrybearblog.com/2011/09/ba sic-macroeconomics.html), it almost doesn’t matter. After all, if the policy response is going to be inappropriate regardless of the size of the data error, having data that makes the problem look smaller only means less of a policy failure in response.

Posted by MikeKimel | Report as abusive

Unfortunately Mike you’re missing the politics in all this. We’ve seen again this week that the Republicans are quite happy to hold their country to ransom just to get their way, putting the successful passage of a bill to provide disaster relief in jeopardy just to enforce their opinion. Standard & Poors were clearly not wrong to downgrade US debt.

I’m not saying the figures were massaged to hide the economic damage of the Bush Presidency, but if the figures had come out earlier it wouldn’t have improved their poll ratings which were already at record lows. All that is necessary for news not to get out is to underfund the agency responsible for producing it so that it comes out more slowly. Delay delivery long enough and you can then blame the next guy for your mistakes.

We’ve also seen recently that austerity alone does not fix debt ratios; in Greece the economy is shrinking faster than the debt because austerity reduces GDP. Insist on too much austerity and the economy collapses, so even though Greek is spending less and repaying more, the debt ratio is actually climbing. The result is an even more damaged economy, less able to repay debt or provide for its citizens.

Posted by FifthDecade | Report as abusive

I fall on the misguided side of this particular debate.

GDP is an aggregate of a bunch of measurements with sizeable error bars. Govt, trade balance, consumption, business investment. Of course it’s going to be pretty wrong pretty often.

Imagine that you were predicting the number of home runs hit by the Yankees for a given season based on prior years’ numbers. Your method is to estimate what each individual player would do and then add them all together to get the team total. Most likely, assuming you don’t have some systematic bias in your guesses that skew your results, your individual predictions would be worse than your team wide numbers. You might overestimate Alex Rodriguez by 20 HRs, but underestimate Granderson by 20.

Now imagine if you made these prediction before finding out that the Yankees moved to a new stadium with the fences all 400+ feet from home plate. Regression to the mean and the use of the past to predict the future, which normally make your predictions work well, no longer are doing you any favors. At that point, your estimates are systematically WAY off on everything. GDP numbers for this recession were like if the Yankees moved to a cavernous new ballpark.

Posted by Some-Guy | Report as abusive

We’re seeing the exact same problem across the spectrum in important gov’t jobs, from public school teachers, to federal regulatory agencies. The GOP has spent the last four decades in a concerted effort to strip away the salaries, benefits, and even the social respect that we used to accord to public servants. Not only are folks now underpaid and overworked (downsizing in the Clinton years did not reduce the amount of work that gov’t workers needed to deal with), they’re regarded by many of their neighbors as parasites.

These are the people who are supposed to teach our children, keep our streets safe, put out fires that threaten our lives and property, haul our garbage away so it doesn’t fester in the streets, keep greedy factory owners from poisoning our food and toys, and on, and on, and on. But Republicans cheer the criminal Rick Scott for laying them off by the thousands.

Still, in telling this story, you should not write out of history the folks who warned us that the stimulus was weak tea.

Posted by Auros | Report as abusive

There was supposed to be a link in that last:
http://krugman.blogs.nytimes.com/2011/09  /27/stimulus-tales/

Posted by Auros | Report as abusive

Of course the statistical challenges aren’t confined to 2008:Q4. We looked at dozens of quarters of estimates. See

http://blog.morphism.com/2011/08/gdp-unc ertainty.html

for 2008.

Posted by JamieStephens | Report as abusive

Felix, what skill set do you believe that economists at the BEA should have? Would you add anyone in particular to the BEA Advisory Committee?

http://www.bea.gov/about/advisory.htm

Posted by STaylor | Report as abusive

One of the problems may be the speed with which the data is supposed to be analyzed and forecasts made. Given that many businesses do not make payments to and/or receive payments from vendors for up to 90 days, it seems to be that it would be pretty difficult to make an accurrate forceast only 30 days after a fiscal quarter ends.

Posted by mfw13 | Report as abusive

The same thing applies to gross domestic purchases, which I have written about a few times. It has been closer to the pulse of how the American economy feels.

http://alephblog.com/2011/03/27/things-a re-not-as-good-as-they-look/

I’ll have another post on this soon.

Posted by DavidMerkel | Report as abusive