What do you do when one line of the latest government statistical release says that U.S. employment grew by 92,000 jobs during October, while 4 paragraphs later the same report gives the number at 437,000?
Given the magnitude of the subsequent revisions, adjustments, birth/death factors, benchmarkings and other statistical sleights of hand, the initial number has become totally marginalized…. [I]t seems foolish to put too much weight onto whether a monthly data point is 90k or 150k or 230k. If it turns out this single number is plus or minus a million per year, then the monthly data becomes worthless.
The Bureau of Labor Statistics collects numbers on how many people have jobs in two ways. The establishment survey asks businesses how many people they had working for them that month, while the household survey sends interviewers to specific residential addresses to find out who at that address claimed to be employed each month. Michael Mandel notes that one of the especially curiouser aspects of the recent discrepancies between the two methods is that the government seems to be having a particularly hard time counting the number of people who are working for … the government:
The number of people reporting that they had public sector jobs …has risen by a very solid 430,000 [since last March]…. However, these numbers come from the household survey, which is based on responses from individuals. The payroll survey, which is based on data from unemployment insurance records, shows a much slower growth of government jobs…only 183,000 since March.
One natural reaction, when faced with such a discrepancy, is to conclude that one of the numbers is right and the other is wrong. Good arguments can be marshalled on either side. For example, Tim Kane observed that the payroll survey misses people who work for newly created establishments (part of the reason for the recent big revisions in those estimates) or are self-employed. Spencer England suggests in a comment left here that the more rapid growth of small firms in recent years might also account for more employment growth being counted by the household survey. On the other hand, Menzie noted that the household survey displays four times the variance, which to me indicates pretty clearly that typically it is much less reliable:
When faced with two different estimates of a magnitude such as the growth in employment, most statisticians would choose not to throw one number out altogether, but rather to combine the two. If the two estimates are unbiased and uncorrelated with each other, the optimal weights are inversely proportional to the variances of the measurement errors. Since the measurement error in the household survey is bigger, it should be given a smaller weight, but that does not mean the number should be ignored completely.
Actually, there’s a third data source that doesn’t get discussed too much, but also may be very helpful, which comes from Automatic Data Processing. This company uses data obtained from payroll services provided for 225,000 businesses in America employing 14 million workers. They release monthly an estimate of what the total job growth for all establishments would be, based on the data they collect.
| BLS payroll |
| Household |
| ADP |
| Weighted |
| BLS payroll |
The table above compares these three estimates, and illustrates the consequences of regarding the initial BLS payroll estimate as 8 times as reliable as either of the other two, that is, using as weights 0.8, 0.1, and 0.1. The resulting “weighted average” column basically nudges the payroll numbers in the direction implied by the average of the other two. It’s also interesting to compare this with what the BLS itself later reported the payroll estimates to be on the basis of new and revised data. The other two series have typically been suggesting recently that employment growth was stronger than the initial payroll estimate implied. If you had bet on revisions in that direction, you generally would have been correct.
If the same pattern continues to hold, we might expect the October payroll figure of 92,000 new jobs to be revised upward in the months to come, with 130,000 perhaps a better estimate than is the initial figure. If that proves to be the case, then combined with the huge upward revisions to the August and September estimates, this is a much better employment report than I had been expecting.
As a statistician, it would also be nice to look for some other corroboration as well. The sharp rise in interest rates immediately following the release of the BLS numbers suggests that Wall Street is seeing this the same way I am: