Felix Salmon

Charts of the day, payrolls accuracy edition

By Felix Salmon
April 7, 2011

Many thanks to Steven Guichard and Zubin Jelveh, who picked up my gauntlet and ran the numbers on whether payrolls numbers released at the beginning of the month are any less accurate than payrolls numbers which come out a bit later.

Here’s a chart from Steven, showing the average absolute change in payrolls numbers between the initial report and the final figure, according to the day of the month that the initial report was released.


And here’s a chart from Zubin, showing the same data in a slightly different form: in this case every revision is a dot. I’m not entirely sure what the outlier is on the right-hand side*, but the message of both charts is clear: there’s no indication at all that payrolls reports released later in the month are any more accurate than those released earlier in the month.


Steven also provided this intriguing graph, which shows that payroll data got significantly more accurate in the 80s and 90s, but seems to be getting less so since the crisis struck.


The average revision seems to be somewhere in the 50,000 range, which is consistent with the official error bars:

The confidence interval for the monthly change in total nonfarm employment from the establishment survey is on the order of plus or minus 100,000. Suppose the estimate of nonfarm employment increases by 50,000 from one month to the next. The 90-percent confidence interval on the monthly change would range from -50,000 to +150,000 (50,000 +/- 100,000). These figures do not mean that the sample results are off by these magnitudes, but rather that there is about a 90-percent chance that the “true” over-the-month change lies within this interval. Since this range includes values of less than zero, we could not say with confidence that nonfarm employment had, in fact, increased that month. If, however, the reported nonfarm employment rise was 250,000, then all of the values within the 90-percent confidence interval would be greater than zero. In this case, it is likely (at least a 90-percent chance) that nonfarm employment had, in fact, risen that month.

It’s important to remember here that there’s an enormous difference, as far as the markets are concerned, between a payrolls number of say 70,000 and a number of 170,000. But each one is within the confidence interval of the other. So the lesson, as I suspected, is that we should treat all payrolls reports with skepticism, but pay no attention to the day of the month they’re released.

*Update: In the comments, Zubin reveals that the outlier dates from December 1995, when the report was set to come out the 5th but was delayed because of a government shutdown. Anybody taking bets on when the payrolls report will be released next month?

2 comments so far | RSS Comments RSS

Thanks felix. The outlier is for the December 1995 report which was set to come out the 5th but was delayed because of a government shutdown. Deja vu all over again.

Posted by zubin | Report as abusive

Um, if you don’t understand the difference between a 70k number and a 170k number with the same confidence intervals, you really shouldn’t be pontificating about statistics. That just shows that you’re an idiot.

Posted by niveditas | Report as abusive

Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/