Be afraid; be very afraid.
From “‘Created or saved’ doesn’t add up”, by Joseph Lawler:
…[t]he “created or saved” numbers are meaningless. The administration purposefully devised the metric to be nebulous. Without a counterfactual, showing the trend of unemployment in the absence of the stimulus, it is impossible to know how many jobs the stimulus saved.
But this is completely counter to what I learned in economics, and how, for instance, the CBO conducts analysis. I assume Mr. Lawler doesn’t dispute the impartiality of the CBO (but who knows?). Here’s the way real macroeconomists conduct analysis:
As the President has discussed, analysis done within the Administration has shown how his tax cuts have substantially offset the series of adverse shocks that have been buffeting the economy. Simulations of a conventional macroeconomic model show that, without the tax cuts, the level of real GDP would have been about 2 percent lower in the middle of 2003. About 1.5 million fewer people would have jobs today. The job market is not what we would like it to be right now, but it would have been worse without the Administration’s actions.
One can view the short-run effects of these tax cuts from a classic Keynesian perspective. The tax cuts let people keep more of the money they earned. This supported consumption and thus helped maintain the aggregate demand for goods and services. There is nothing novel about this. It is very conventional short-run stabilization policy: You can find it in all of the leading textbooks.
The writer is Greg Mankiw, discussing in 2007 a particular fiscal measure, namely the 2003 tax cuts (h/t, Brad Delong).
So let us return to how the Congressional Budget Office (CBO) conducted analysis. In their February analysis, they presented this set of results, based on a range of multipliers in the literature.
Table 1: from CBO, Estimated Macroeconomic Impacts of H.R. 1 as Passed by the House and by the Senate, February 11, 2009.
So GDP is estimated to be between 1.4 to 3.8 percentage points (ppts) higher than baseline in 2009Q4, due to the stimulus bill. The midpoint of this range is 2.6 ppts. Relatedly, the range of employment gain relative to baseline is 0.8 to 2.3 million; the midpoint of this range is 1.55 million.
Interestingly, taking the CEA’s model based approach (Table 2, Joint Economic Committee testimony of 22 October), and assuming the same incremental growth rate in 09Q4 as in 09Q3, the implied deviation from baseline is 2.56 ppts, or right in the midpoint of the CBO’s range.
Now using the error correction model that I estimated in last Tuesday’s post (where the cointegrating relationship between log GDP and log nonfarm payroll employment is 0.37), I find the range of increased employment relative to baseline is between 0.68 and 1.84 million, slightly lower than the CBO range of 0.8 and 2.3 million. The estimated employment impact is 1.26 million, using the midpoint of the CBO range for impact on GDP.
I know counterfactuals and math are hard to fit on a bumper sticker. But one would hope that in an 800-plus word essay on economics (even if in Politico), some economic content could be included.
By the way, Jeff Frankel debunks a similar misapprehension in National Journal.
Let’s tell it like it really is. Both tax cuts then and stimulus spending now have increased gov’t debt.
Gov’t debt has helped maintain the aggregate demand for goods and services.
Here is better yet. If the domestic and foreign rich can’t get the lower and middle class further into debt to them, the domestic and foreign rich will get the gov’t to do it for them.
Doesn’t the same graphic show that the midpoint between the low and high estimates for unemployment with the plan at 8.2%?
Since the official numbers just crested 10.2%, why would one use the information in this report to calculate the number of jobs “created or saved” when it underestimated the entire unemployment rate by a full two percentage points?
Great stuff as usual. I will be using this soon.
I made such a similar argument re. “counterfactual” in “Outside The Beltway” comments that I feel like Lawler cribbed my notes.
We know that economics teaches ways to deal with the missing counterfactual. It is the “all things being equal” gambit. That is, if there have been such and such multipliers in the past, then all thing being equal, they should work again today.
How do we know? Your expansion from a number to “ranges” is good, but does the discipline of economics have a way to assign confidence bars?
I don’t think so. I think the confidence is self-referential, based on all things being equal.
How useful or informative are these simulations ? Are they not based on a historical data set that does not include any comparable intermediation failure such as we have experienced? The Lucas critique seems a propos.
The problem with “counterfactuals” is that they are not factual. They are estimates/forecasts/projections/assessments/whatever based on assumptions and methods that may or may not be correct–whether they are conventional or not. While I do not question the bias of any of the sources you use here, that does not mean that they have presented any facts.
It may not be astute quantitative economic analysis, but I do think Politico has a point: No one really knows how many jobs have saved or created by the stimulus program. Moreover, the opportunity and incentives for shenanigans in the Administration’s “tracking” of these new/saved jobs are massive.
BTW, I am not a big stimulus opponent. I actually accept that some sort of stimulus was necessary. I just think we have to fall back to simpler claims. We can measure billions injected by simple accounting. We can count direct-hires under stimulus programs as well.
If I’d had my druthers, we’d have thought of stimulus as more a time-shifting effort, bringing forward and accelerating necessary government spending, rather than an invention of make-work.
A smart grid works for me as a time-sifting effort, because I think we’ll need it.
Funny you should be quoting Mankiw….at
http://gregmankiw.blogspot.com/2009/11/taking-out-trash.html
he takes issue with Krugman taking issue with him taking issue with Administration claims on job creation.
Dr. Chin’s point is not that simulations are facts. He is trying to point out that Politico’s criticisms of the Obama administration are not valid. The current estimates of jobs saved or created are no different than the same kind of estimates produced in previous administrations. And absent an ability to travel to a parallel universe without stimulus spending, these estimates are the best we can do. Given that actual unemployment is significantly higher than originally estimated, it might make sense to update the simulations, and also account for more direct data of what has been spent in stimulus as well as direct counts of job creation. But I doubt an updated simulation would significantly change the original estimates.
But MarkOhio, doesn’t it become a pissing contest with everyone choosing parallels in history to suit their ideology?
In other words, “this number is as good as any” isn’t quite the same as “this number is valid.”
Dr. Chinn, my article was about messaging, not economic analysis. I do not claim to be a “real macroeconomist” or anything other than a journalist. I’ll have a response to your criticism soon.
Odograph, where were your original comments? I didn’t crib from your notes (or anyone else’s) but I would be interested in seeing them now.
Menzie-
Since the CBO forecasts were obviously wildly wrong, why give them any credence? Of course, they may not have factored in the serial stupidity of this administration in one job-killing measure after another. Cap and tax, card check, auto bailouts, nationalized health care, compensation caps, etc.
It is amazing that the forecast could be so far off over such a short time frame. As far as tax cuts go, didn’t Ms. Romer produce a paper indicating that tax cuts (more properly cuts in tax rates) were quite powerful in stimulating output growth?
Hi Joseph, I suppose this thread is a reasonable example:
http://www.outsidethebeltway.com/archives/now_were_in_a_position_to_really_acclerate/
I suppose I should remember that the internets are a big place, and probably more than one of us are having the same thought.
“But this is completely counter to what I learned in economics”
Menzie, you know far more about statistics than your audience on this blog, but you seem to consistently be over confident in your estimates.
The uncertainty around any estimate of jobs created make both the administrations claims and the claims of the critics possible.
We will never have conclusive statistical proof that the stimulus worked. In about three or four years, we will be able to make an educated guess.
Joseph Lawler: You write, without caveats, quote:
That seems like an unambiguously economic assessment to me.
I followed Simon van Norden’s link above to Mankiw’s blog related to the topic of this debate – http://gregmankiw.blogspot.com/2009/11/taking-out-trash.html. There Mankiw is complaining about a cheap shot from Krugman, mischaracterizing one of his earlier blogs. Mankiw’s proposed linguistic distinctions between what is measured and what is econometric modeling based estimates seems reasonable to me. But then Mankiw gives an example of his beef with how the Obama administration is using the language, pointing us here: http://www.recovery.gov/Pages/home.aspx
But it turns out that the way they are generating the jobs saved number on the page Mankiw is complaining about doesn’t involve any modeling at all. They are rather just asking employers who received funding how many worker hours they used the funding to pay for: http://www.recovery.gov/FAQ/Pages/ForCitizens.aspx#jobscreatedsaved
I feel much better when the claim “Jobs ‘created or saved’ is meaningless” comes from a real economist, like Ed Lazear. (One can also quote Mankiw directly, rather than roundabout through DeLong).
And also when reports that starkly criticize the “save jobs” numbers come from sources more reputable than Politico, such as the Financial Times…
~~ quote ~~
… Many universities, for example, are including tenured academics in their jobs created and saved numbers even though their jobs were already guaranteed for life.
Ms Smith, who is associate vice-chancellor for research administration at the University of California, Los Angeles and leads a team handling its stimulus awards, received guidance from the UC Office of the President saying she should include everyone paid by stimulus dollars, including tenured faculty members.
…it did avoid a very sticky problem: how can you know for sure whether a job would have disappeared were it not for stimulus money?
[U.C.? Hmm….]
~~~
… or the NY Times-owned Boston Globe…
~~ quote ~~
While Massachusetts recipients of federal stimulus money collectively report 12,374 jobs saved or created, a Globe review shows that number is wildly exaggerated…
~~~
… and the AP…
~~ quote ~~
Salary raises counted as saved jobs; 2 out of 3 Head Start jobs overcounted
An Associated Press review of the latest stimulus reports … found that more than two-thirds of 14,506 jobs credited to the recovery act under spending by just one federal office were overstated because they counted pay increases for existing workers as jobs saved.
The inflated job count is at least partly the product of the administration instructing local community agencies that received money to count the raises as jobs saved…
But officials defended the practice of counting raises as saved jobs. “If I give you a raise, it is going to save a portion of your job,” HHS spokesman Luis Rosero said.
Most of the inflated figures were like those cited … by Southwest Georgia Community Action in Moultrie, Ga. The agency, like hundreds of others collecting Head Start money, claimed all its existing employees’ jobs were saved because they received a pay raise with the stimulus cash…
The agency employs 508 people but claimed 935 jobs were saved.
… director Myrtis Mulkey-Ndawula said she followed the guidelines the Obama administration provided. She said she multiplied the 508 employees by 1.84 the percentage pay raise they received and came up with 935 jobs saved.
~~~
It’s reassuring that relevant federal government officials actually officially on-the-record defend the practice of counting raises in pay as saved jobs, eh?
(Though as to the math used, need I point out that Head Start, which multiplied 508 times 1.8% to get 935, is an educational agency?)
So I’m now happy and relieved to realize that when NYC’s transit workers union took $350 million in stimulus money for an 11% raise in pay — which I previously thought would keep that money from creating any new jobs — it actually saved 33,000 jobs, as calculated by U.C. Or, as the Head Start people would calculate it, 363,000 jobs!
Jim Glass documents the sort of shennanigans that can happen when people are asked to directly count jobs saved or created. This is why economists often prefer to use simulations (driven by macroeconomic data gathered for non- or at least less political purposes).
Odograph, I suppose dueling simulations are a kind of “pissing contest.” But they have an advantage over other sorts of “pissing contests.” The methods they use are open to examination and critique by others (we can all understand how the numbers were calculated). And they are more based in reality (at least historical data) than the erroneous direct counts of job creation. Plus you can use them to produce a range of plausible numbers, similar to the CBO’s original estimates. When we are talking about a no-stimulus version of history that we can never observe, it shouldn’t be suprising that reasonable people disagree (or engage in “pissing contests”). A simulation can be used to set the boundaries of what counts as reasonable disagreement.
I am not suprised that CBO’s Feb09 estimates of unemployment are off the mark. Given that 10%+ unemployment is rarely observed in the historical economic data, a simulation model should not be expected to forecast an expected outcome of 10%+ unemployment. The forecast error just shows that the current downturn is unusually severe. BTW, I would guess that most of the economic models would predict a larger stimulus impact under higher rates of unemployment (less crowding out with higher unemployment).
Dr. Chinn,
When I wrote “There’s a problem, though: the “created or saved” numbers are meaningless,” I was referring to the numbers that administration has been touting recently, specifically to the claims that Joe Biden and Jared Bernstein made on October 30th regarding the new reports based on stimulus recipient responses. Those claims are different from a straightforward analysis using standard models.
If the administration wants to announce that their models predict 3.5 million jobs will be created or saved, that’s fine. But if they make it a central focus of their messaging that they have saved 650,000 jobs and that they know that from a review of the new data, I take issue.
My full response is here: http://spectator.org/blog/2009/11/13/destroyed-or-prevented
Note that I think my point stands, but that I am open to amending my argument if necessary.
Jim Glass: The hyperlink I provided is direct to Mankiw, not to DeLong. Please click through before committing virtual pen to virtual paper.
By the way, feel free to criticize the tabulation approach (as opposed to the model based approach — see MarkOhio‘s comments); that in no way affects my argument. I believe that tabulations will undercount the total number of jobs created by the stimulus since it excludes tax-policy-induced changes and post-impact multiplier rounds.
Joseph Lawler: Thank you for your comments. From WaPo:
So, if you inspect the second paragraph, it is clear that they are referring to the model/projections based estimates from the CEA, and not the Recovery.gov data only for their 1 million estimate. This confusion regarding numbers often comes up when people misinterpret the Recovery.gov data (see for instance Richard Posner’s difficulties). Hence, my point still stands — that the mode of analysis is entirely consistent with the application of changes relative to counterfactuals.
I will also observe, as have some of the commenters, that the distinction Professor Mankiw makes in his more recent “Taking out the trash” post seem highly nuanced and semantic in nature. I am, however, in agreement with Mankiw that we will not be able to provide precise estimates, so I will continue to try to provide ranges and mid-points of ranges, and standard errors (which I believe is the right way to conduct policy analysis).
Notice that in the later Bush years Dr Chin used the full cost of the campaign including future interest on debt, compounded by the assumption that the marginal war dollar was a marginal debt dollar. Here we use the simple outlay instead.
KevinM: If I were tyring to assess the benefit-cost ratio for the $787 billion stimulus bill, I would want to compare the inflation adjusted cumulative debt incurred against the integral of the area under the potential GDP line and above output, discounted appropriately. But I’m not. I’m just trying to defend the exercise of (1) calculating a reasonable impact of tax cuts and government expenditures on real GDP, and then (2) relating that change relative to baseline to an implied change in employment, based on historical correlations. So, I don’t want to do the comparison you mention.
I really look forward to watching you attempt to justify this administration’s economic policies for the next year or three.
As to the CBO, Martin Feldstein had this to say, “The Congressional Budget Office is required to estimate the cost of the law as it is written, not as it may evolve. But we as taxpayers will have to pay those future costs.” He had an excellent column recently about the proposed federal takeover of our health care system and why it won’t work.
I think the biggest problem Keynesian macro-economists have in forecasting is that the darned economy just refuses to remain static. Hayek spoke about this problem at some length. In the unlikely event you ever perfect a macro model, it will be obsolete and useless anyway by say, the next day.
Hitchhiker: Thanks. Similarly, I look forward to reading your your exegeses on Hayek and
your analyses like this:
Jeffrey Sachs has an interesting commentary in the FT, criticizing Obama and the ARRA from a modern Rooseveltian / European-style social democrat perspective:
http://www.ft.com/cms/s/0/15c18868-ce2f-11de-a1ea-00144feabdc0.html
I’m more of a European-style liberal, which is on the right economically, but I wouldn’t be disappointed with Obama if he were successfully pursuing the kind of vision that Sachs lays out. Whatever Americans want and vote for is fine by me. I’m not the sort of purist libertarian who insists that all public works are doomed to fail. They can and many do work fine, but they require political and public administrative skills that the US hasn’t tried much to develop.
The ARRA shows that. Even those who wanted Obama to boost spending are upset that he allowed Congress to push through such a disorganized grab-bag of aimless pork. The only half-serious argument in the ARRA’s defense is the neo-Keynesian creed that any kind of government spending during a recession is good. For those of us who can’t make that leap of faith, we see a populist bill that cushioned employment and production in the short-term at the expense of long-lasting inefficiency and financial rent.
In Obama’s defense, he was to some extent led into a trap by Bush, who dropped all pretense of fiscal conservativism in his final months and threw money around like there was no tomorrow (for him there wasn’t). As a democrat Obama was under a lot of pressure to out-spend Bush.
But Obama charged right into that trap, and caved in to Congressional porkers and Wall Street party sponsors far more than he needed to. And maybe since neo-Keynesians were the only economists who could justify what he was doing, he surrounded himself with them. And so his presidency is already in deep trouble less than a year in.
By the way, I have no sympathy whatsoever for the mostly racist kooks who want Obama to fail, and I’m sure Sachs doesn’t either. Obama is obviously a much smarter man than Bush was. I hope he wises up.
Dr. Chin writes above:
“…so I will continue to try to provide ranges and mid-points of ranges, and standard errors (which I believe is the right way to conduct policy analysis).”
I understand that we need to know what these data points are, but I think we need to be very careful in how we use this information.
One thing we have experienced at least financially, if not more broadly economically, is the proverbial “black swan”–a very fat, far out tail as measured by any statistical tool. As a result, I think it is dangerous to focus on “normal” probabilities/averages going forward, at least until we get most of the way out of this mess.
To borrow a phrase, it is different this time, but not in the way the Reinhardt & Rogoff meant.
I think it would be unwise either as an economist or a policy maker to expect the jobs picture to return to normal based on either historical averages or estimated ranges, mid-points, and standard errors using conventional methods.
I think there may be two better ways to approach the problem:
1. Rely on the estimates of those whose recent estimates have proven most correct in the current great recession. Who has more accurately predicted the trend in unemployment/employment? What is unusual or unique about their methods that led them to be more correct? This may include a small sample of prognostications averaged (maybe even weighted by accuracy) or even only one.
2. More in line with your approach, use averages, etc., from historically bad finanically-driven recessions with massive unemployment–not all economic slumps. In this way, we may be comparing “like” events (or at least “more like”) rather than throwing all of economic history into a single statistical basket.
There may even be a way to integrate these two approaches: Use the “best” current methods (ie–most accurate) to project from “like” history.
Otherwise, I think you’re doomed to, at best, economic analysis and policy making mediocrity, likely understating policy needs by overstating expected job recovery.
Terry: Your point regarding alternative approaches is well taken. In fact, so well taken that I actually did mention the second approach (averages over post-crises periods) in several of my previous posts, including this one the last IMF WEO.
Perhaps the administration should keep its powder dry next time and not issue language about saving (or later “saving or creating”) a certain number of jobs. Nor sure it pretend that it will demonstrate these jobs. Realism in economic journalism would be nice. Realism in economic rhetoric from the country’s leadership would be just as nice.
I stand by my analysis. Quite short and pointed and not endowed with pages of academic explanation and hedging but, essentially correct, in my opinion, of course. The stock market crash of 1929 had nothing to do with the depression causally. The financial meltdown in 2008 had nothing causally to do with our current recession but, was a result of it. Excessive risk taking is hidden as long as the market continues up, in this case housing prices. It is only when the market decreases that the excesses become readily apparent.
It is my belief, that you are a very partisan person politically, not unlike Paul Krugman. Perhaps I am wrong. But, you appear to defend the administration when no defense is possible. You have defended climate change science when there is nothing scientific about it at all. Be careful. Every new piece of legislation dreamed up by Obama or Pelosi will cause any economic model to need recalibrating or be trashed completely.
There can be no economic justification for what we have seen so far coming out of DC. Will you support another stimulus of Acorn, Indian tribes, government employees, etc when the unemployment level passes 12%?
I suppose you might support the march towards socialism that began in 1930 and received a big boost recently since economic models would be in much demand and the planning inherent with such schemes would cause people like you to become in high demand. For many, the only reason our current crop of economic remedies have failed to work is the government lacks the power to coerce the market participants to do what they want them to do. We need more regulation. More planning. It is a very slippery slope we are careening down.
In any 10 year period in the history of the United States, has there ever been a period where the number of working americans grew so slowly? I don’t have job numbers for the great depression, but if you look at the recessions of the 70’s, 80’s and 90’s, strong population growth lead to job growth.
Our population just is not growing the way it used to. Econ 102 talks about capital and labor. The growth in supply of labor is down, the demand for labor is down. Where is economic growth headed?
Hitchhiker: I applaud your consistency. In other words, you’re saying “Road to Serfdom II”.
http://thedanashow.wordpress.com/2009/11/16/stimulus-spent-over-617k-in-non-existent-missouri-district/
Yep, money well spent…
Yes indeed. I am. I applaud your mathematical abilities but, I question the underlying models upon which most of your calculations are done.
Will knowing physics and being able to quantify the forces and angles, vehicle weights, etc. involved in any given car crash help the driver avoid the crash?
The people involved in setting the economic policy use economists as cover not as objective advisers. Their goal is not full employment or increased growth, it is simply political power.
I see you as someone that helps the socialists continue the consolidation of political power by elevating economics as a science that can be used to coerce outcomes rather than as a science to merely explain results and therefore you are an enemy to freedom. Nothing personal.
Hitchhiker: Thanks for the compliment, but I’m just using high school (at most) math in this post.
Actually, I think knowing what momentum is, what kinetic energy is, and crumple zones are (that’s engineering vs. physics, I admit) are all useful for avoiding or mitigating injury. This much I’ve learned from my friend who deals with highway safety issues. So I respectfully disagree with your argument.
So I see you as a person pursuing verbal McCarthyism. Nothing personal.
I agree that the “created or saved” numbers are dubious. Simply put, spending can simultaneous create and save, and also destroy, jobs at the same time. And, econometric measures, particularly in the medium to long term, aren’t very accurate.
The CBO and White House Budget office (and every other such body known to man) has a reliable record of making inaccurate forecasts in the medium to long term. This is no slight to the hard working professionals there. They do their best to ascertain numbers that budget makers need to make assumptions about in order to function. Some estimate is better than no estimate. If you can’t get an estimate, you can’t adopt a budget and government comes to a screaching halt.
But, the margin of error grows greatly as you get farther out, and the accuracy of these estimates is fuzzy in the best of times. Of course, we are not in the best of times now. Current conditions are outliers from the data that were used to create the models in the first place, so the models are at their least reliable point right now.
Macroeconomics is a bit like weather prediction. Forecasts for anything other than very short time periods (the moral equivalent of five day forecasts) and very long time periods (the moral equivalent of climate change) are inherently hard to make accurately. Also, the models aren’t continuous. The factors that matter for each kind of weather prediction (short, medium and long range) are very different. If you want a five day forecast, you look at satellite images and barometric pressure trends over lots of data points. If you want Hurricane severity forecast for the SE United States over the next year, you look at the rainfall in Africa in the most recent rainy season and Atlantic Ocean surface temperatures. If you want a hundred year weather forecast, you look at ice core samples from glaciers and tree ring data.
Jobs saved or created fits firmly in the realm of medium term econometric predictions of complex phenomena that economic models simply are not very good at producing accurate results.