The Heritage Foundation critiques the CEA assessment of the stimulus. In WebMemo #2799, Dr. Campbell writes:
The CEA’s method, in brief, compared a statistical forecast of the economy based on historical patterns (no stimulus) with the actual economic results in 2009. On this basis, it claims that there are 2 million more jobs in the economy than otherwise would have been the case. The CEA then concludes that this difference between this statistical forecast and the actual results were the effect of the stimulus.
Yet the CEA’s benchmarks for unemployment and GDP numbers were completely arbitrary. If the Administration had used other economic forecasts, the results would not have been as impressive–in fact, some would have shown that the economy lost more jobs after the stimulus package was implemented.
One way to see the inadequacy of the CEA’s method is to compare it with other economic forecasts made for 2009–before details of the stimulus plan were known. These forecasts were done by companies and agencies that have a direct interest in making the most accurate forecast possible so that businesses and governments can plan accordingly.
Well, I am confused on a number of counts.
- First, the author selects forecasts from well before the ARRA was passed — October 2008 for the Global Insight forecast and January 2009 for the CBO (although that forecast was finalized on December 19). But, as I pointed out in this post one has to condition for shocks that occur between the time of the forecast for a baseline, and the ex post event. Nobody disagrees (I hope) that the consensus forecast deteriorated substantially between December and February, when the ARRA was finally passed (see the deterioration in the WSJ mean forecast in this post).
- Second, the CEA 2nd quarterly report clearly shows the estimated impacts on GDP growth rates from Global Insight, as well as other Macro Forecasters. These implied increments to growth rates do not jibe with the the inferences drawn by Dr. Campbell — that the impact on GDP is much smaller than CEA asserts when using forecasts from the other agencies and firms. In fact, the NYT compiled an interesting graph (reproduced here and below), showing the counterfactuals undertaken by the other firms.
Source: J. Calmes and M. Cooper, “New Consensus Sees Stimulus Package as Worthy Step,” NYT (Nov. 21, 2009).
CEA summarized the estimates in their report — see Table 7, reproduced in this post, and below.
Table 7: from CEA, “The Economic Impact of the American Recovery and Reinvestment Act of 2009: Second Quarterly Report,” (January 13, 2009).
- On a minor point, for the life of me, I cannot figure out how the numbers reported in this figure from the article were generated. I only discovered this by trying to replicate the various series, and discovering that they’re all nominal GDP figures. But the CEA reports did not provide nominal GDP figures, so I’m at a loss at figuring out how the graph was constructed.
Dr. Campbell continues:
The economic impact of the fiscal stimulus bill must be evaluated by projecting the economy without the stimulus bill and then introducing the fiscal stimulus to that same forecast. The CEA’s report constructs and then analyzes a forecast of a downward spiraling economy. It runs a “what if” scenario in the wrong direction. Rather than analyzing the economic impact of the fiscal stimulus on a benchmark forecast, the CEA constructs a forecast and benchmarks it to what actually occurred.
I think I must’ve read a different study than she did. CEA did run a counterfactual approach — it was a VAR approach. As I noted when I replicated CEA’s VAR approach in this post, it’s true the deviation from forecasted amalgamates fiscal and monetary policies and monetary and financial shocks, but that is a baseline one could use. But the resulting implied effect is consistent with the multiplier (or “model”) approach used elsewhere in the study, so I’m not certain what the criticism is.
Dr. Campbell concludes:
The CEA claims that the stimulus bill created jobs in 2009, but this claim is based on its newly constructed “it would have been worse” forecast for 2009. When trillions of dollars are being spent, the American people deserve to have a true economic analysis done and should not waste money on meaningless reports.
I guess “true economic analysis” is in the eye of the beholder.
[Update, 4:35 Pacific]
Dr. Campbell has posted a response and comments (below). Here is my rejoinder to her comments.
Let me thank Dr. Karen Campbell for her response. Let me address several points in turn.
First, if the conclusion is that any agency that uses the “model” approach used by CEA, then we should close down the CBO. In particular, see this report. I think this puts the criticism is its proper context.
Second, to drive home this point, we should also put in the category of agencies and individuals doing “bad economics” people like Stanford Professor of Economics John Taylor (and former Undersecretary of the Treasury under G.W. Bush), and the modelers cited in this blogpost. See my take on his point here.
Third, to the substance of the matter, regarding baselines. I agree, it would be best to use a forecast just before the policy change is implemented, then compare outcomes to pre-shock forecast. For institutional reasons, agencies are not able to report such results typically because official forecasts are not made on a constant basis. Rather, two official forecasts are conducted each year, and are the product of a consultative process by the Troika (Treasury, CEA, OMB).
Now, there very well might be internal forecasts, but because of the aforementioned institutional constraints, those cannot (in my understanding) be reported in an official US Government document. I presume (but do not know) that CBO has a similar stricture. (Full disclosure: I was on CEA staff 2000-01 in Clinton/Bush Administrations; I was visiting scholar at CBO Macroeconomic Analysis Division in 2005).
When I was in the government, the Macroeconomic Advisers macro model was used as a workhorse model for evaluating, in-house, various policy scenarios. If this is still true (and I have not asked my contacts in the USG if it is), then one can guess that the baseline should look very similar to whatever no-stimulus baseline Macroeconomic Advisers has, with accounting for whatever other assumptions are relevant (including rest-of-world GDP, price of oil, etc.).
In the end, I would say in an ideal world, we should hope to do something like what you propose. In the real world, confronted with data inadequacies and (for the government agencies), constraints on what can or cannot be reported, what is undertaken by CEA and CBO seem quite appropriate. (So, I think CBO should not be closed down.).
By the way, since you were gracious enough to respond to my post, I would welcome elucidation on how the series in the Figure in your article were constructed/obtained.