## Charts of the day, payrolls accuracy edition

Many thanks to Steven Guichard and Zubin Jelveh, who picked up my gauntlet and ran the numbers on whether payrolls numbers released at the beginning of the month are any less accurate than payrolls numbers which come out a bit later.

Here’s a chart from Steven, showing the average absolute change in payrolls numbers between the initial report and the final figure, according to the day of the month that the initial report was released.

And here’s a chart from Zubin, showing the same data in a slightly different form: in this case every revision is a dot. I’m not entirely sure what the outlier is on the right-hand side*, but the message of both charts is clear: there’s no indication at all that payrolls reports released later in the month are any more accurate than those released earlier in the month.

Steven also provided this intriguing graph, which shows that payroll data got significantly more accurate in the 80s and 90s, but seems to be getting less so since the crisis struck.

The average revision seems to be somewhere in the 50,000 range, which is consistent with the official error bars:

The confidence interval for the monthly change in total nonfarm employment from the establishment survey is on the order of plus or minus 100,000. Suppose the estimate of nonfarm employment increases by 50,000 from one month to the next. The 90-percent confidence interval on the monthly change would range from -50,000 to +150,000 (50,000 +/- 100,000). These figures do not mean that the sample results are off by these magnitudes, but rather that there is about a 90-percent chance that the “true” over-the-month change lies within this interval. Since this range includes values of less than zero, we could not say with confidence that nonfarm employment had, in fact, increased that month. If, however, the reported nonfarm employment rise was 250,000, then all of the values within the 90-percent confidence interval would be greater than zero. In this case, it is likely (at least a 90-percent chance) that nonfarm employment had, in fact, risen that month.

It’s important to remember here that there’s an enormous difference, as far as the markets are concerned, between a payrolls number of say 70,000 and a number of 170,000. But each one is within the confidence interval of the other. So the lesson, as I suspected, is that we should treat all payrolls reports with skepticism, but pay no attention to the day of the month they’re released.

*** Update**: In the comments, Zubin reveals that the outlier dates from December 1995, when the report was set to come out the 5th but was delayed because of a government shutdown. Anybody taking bets on when the payrolls report will be released next month?