Payrolls and error bars

By Felix Salmon
August 3, 2012
jobs report out, and an absolutely enormous number of people is puzzling over what it might mean.

" data-share-img="" data-share="twitter,facebook,linkedin,reddit,google" data-share-count="true">

It’s the first Friday of the month, which means there’s a new jobs report out, and an absolutely enormous number of people are puzzling over what it might mean. And this close to a general election, there’s also lots of politicians trying to make hay out of it. Both of which, for the intellectually honest, are largely futile exercises at the best of times, and especially so this month.

There are two components to the jobs report: the household survey, which polls households to measure how many people are unemployed; and the establishment survey, which polls employers to measure how many people have jobs. The establishment survey is the more accurate of the two; the household survey is the one politicians gravitate towards. Taken as a package, the two together sometimes convey substantive information on how the economy is doing, and the speed with which we are approaching — or moving away from — full employment.

This month, however, not so much. And in general, the amount of useful information in the monthly jobs report is much lower than the amount of attention paid towards it would imply. Every month, policy wonks and data-heads pore through the 25 different tables in the report, sometimes looking at their own favorite number (U-6 is particularly popular these days), and sometimes just looking for something to jump out at them. (Look at what happened to youth unemployment among Hispanics!)

The latter exercise is particularly dangerous. When you have 25 tables, each with hundreds of datapoints, it’s a statistical certainty that one or two of them are going to be weirdly off in some way. If you then pull out that datapoint and read too much into it, you’re guilty of a logical solecism, a bit like two people who discover they share a birthday and then exclaim at how unlikely that is.

But even the simple check-out-this-headline-number exercise, which every major news organization goes through every month, is statistically dubious. The headline payrolls number has error bars which are more than 100,000 people wide in either direction: if the number everybody’s quoting is 163,000, for instance, all that really says is that there’s a 95% chance that it’s somewhere between 59,000 and 267,000. What’s more, there’s a good 5% chance that it’s outside that wide range. Take the last two years of payrolls headlines, and the chances are that somewhere in there, a number is more than 100,000 off.

And the unemployment rate is less accurate still. Most people will say that it ticked up this month, to 8.3%; the official news release is closer to the mark in saying that it was “essentially unchanged”. And if you look at the numbers it’s based on, they’re all over the place.

This month, the two parts of the report tell differing stories: the headline payrolls number was higher than most people expected, even as the headline unemployment rate went up. Look a bit more deeply, and things become ever messier, what with all the revisions and the changes in people looking for work, and the decrease in median unemployment duration, and so on and so forth. As a result, this month of all months it’s actually quite easy for the markets (if not the politicians) to move on to the next thing. After all, they have very short attention spans at the best of times.

But the fact is that even when the stars align, and the two parts of the jobs report tell the same story, and there seems to be a clear message shining through — even then, there’s nearly always much less to the report than meets the eye. All those numbers will be revised in subsequent months, and the message is never as clear as the media and the markets make it out to be, and in any case the chances are that we’re not actually seeing signal in the noise at all, but instead that we’re just being fooled by randomness.

So be happy, this month, that you can’t work out what the jobs report is trying to say. Because that’s probably exactly the right conclusion to draw on the first Friday of every month.

3 comments

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/

Household survey numbers aren’t revised.

Posted by kuiper | Report as abusive

I thought the purpose of economic statistics was to support political preconceptions? How does reality matter?

Posted by TFF | Report as abusive

kuiper, they are revised. Just once a year.

Posted by KevyD | Report as abusive