Or, the self-rehabilitation effort continues. In a WSJ op-ed entitled “Government Forecasters Might as Well Use a Ouija Board”, he writes:
My analysis of 1999-2013 reveals that the CBO’s real GDP growth forecasts for the next year were off, on average, by 1.7 percentage points, either too high or low. Administration forecasts were similarly off by a slightly larger 1.8 percentage points on average, also to high or too low. Given that the average growth rate during this period was only 2.1%, errors of this magnitude are substantial.
Perhaps most damning: History is a better predictor of annual growth than government forecasts. Simply assuming that GDP growth will be 3.1% in each year — the average annual rate for the 30 years that precede the study period — results in an average forecast error of 1.5 percentage points.
This is an interesting conclusion, and I don’t have doubt that one could pick a number such that a no-change growth rate prediction could outperform forecasts. But it’s not clear this finding is robust. It’s also not clear it is a relevant comparison. Table 1 from CBO (2013) presents some information regarding forecast errors (for two year horizons).
Source: CBO, CBO’S Economic Forecasting Record: 2013 Update (January 2013).
Notice the bias and dispersion are similar across government and private sector forecasts. CBO in particular is on par with the Blue Chip. Note that RMSE and MAE seem roughly equivalent; MAE is more appropriate if the distribution of forecast errors is non-normal).
Lazear’s criterion is focused on the second moment over a particular sample period. However, one might reasonably believe that forecasting accuracy is more critical at turning points (peaks, troughs). The CBO provides an interesting comparison of errors associated with CBO, OMB and Blue Chip forecasts.
Source: CBO, CBO’S Economic Forecasting Record: 2013 Update (January 2013).
Notice the largest error in 2008 is that of the Administration. Relatedly, one of Ed Lazear’s earliest appearances in Econbrowser is in this 2008 post, wherein he is quoted from a May 8th WSJ article:
“The data are pretty clear that we are not in a recession.”
This was, and remains, a surprising comment. After all, 23 respondents in the May 2008 WSJ survey forecasted negative growth for 2008Q2.
Figure 1: Quarter on quarter SAAR growth forecasts for 2008Q2, from Wall Street Journal May 2008 survey. Source: WSJ.
Interestingly, Liberty Street/FRBNY reports that about 45% of the research staff in April 2008 (I’m eyeballing the third figure) expected negative growth over 2007-08.
Now, readers can judge for themselves whether an unconditional no-recession statement was reasonable. To assist in your assessment, I present the data Ed Lazear knew as of May 8, 2008, relying upon the St. Louis Fed ALFRED database.
Figure 2: Log nonfarm payroll employment (blue), industrial production (red), real retail sales (green), and real disposable income excluding transfers (purple), all normalized to zero at 2007M12; data available as of May 8, 2008. Source: St. Louis Fed ALFRED database, and author’s calculations.
I don’t have the Macroeconomic Advisers monthly GDP series that Professor Lazear would have had, but here is a WSJ RTE post from three weeks before Professor Lazear’s statement:
Forecasting firm Macroeconomic Advisers updated its monthly GDP estimate for February, showing a 1.2% decline in February. It was the second largest one-month decline in the nearly 16-year history of the index, behind a 1.6% drop in September 2001.
“The sharp drop in monthly GDP in February was led by a sharp decline in nonfarm inventory investment,” said Macroeconomic Advisers. “Also subtracting from monthly GDP were declines in net exports, capital goods, and construction.”
The firm currently expects GDP to grow 0.2% in the first quarter. The monthly GDP estimate is one of the estimates examined by the National Bureau of Economic Research when it is calculating when a recession has begun.
For a post soon: a tabulation of who predicted in 2008-09 massive inflation as a consequence of unconventional monetary policy…
For more on government agency forecasts across countries, see Frankel, OREP (2011). For other criterion in forecasting, in an exchange rate context, see Cheung et al. (2005). The direction of change criterion in particular seems useful (see Stekler (2011), p.116), but has not been applied in the context of government agency forecasts of GDP, with a few exceptions (e.g., Sinclair et al. (2010)).
Update, 10/28, 6:30PM: Reader Vivian Darkbloom criticizes me for focusing on 2008Q2 forecasts, but not the 2008Q3 forecasts from the May 2008 WSJ survey. First, note the 2 consecutive quarter recession criterion is merely a rule of thumb. Second, nine respondents out of 51 reported average negative 2 quarter growth. Third, the median probability of recession in the May survey is 65%.
Source: WSJ May 2008 survey.
I think that’s a relevant statistic.
Menzie wrote:
“This is an interesting conclusion, and I don’t have doubt that one could pick a number such that a no-change growth rate prediction could outperform forecasts. But it’s not clear this finding is robust.”
This is a startling admission. So essentially, Menzie, you are admitting that forecasts are worse than no forecast at all, but you prefer a forecast because it is “robust” though less accurate. This is an unusually candid self-evaluation of your motivation.
So that would lead us to conclude that policy changes based on less accurate data – forecasts – would create greater errors than policy changes based on historical data.
Then, why not take it even farther? Policy changes should be based on returning decision making to the market traders, the ones with the knowledge of their businesses, and getting the government bureaucrats out of the crony manipulation of business.
Now that is an excellent idea!!!
Ricardo: Do you know how to calculate an RMSE? Geez. I can find a constant drift forecast that might work for one sample, but if it doesn’t work for another sample as I add in data, then it’s not really a “finding”. To protect against data mining, at a minimum one would need to use a rolling or recursive regression to update the drift parameter….
We are comparing apples to oranges. Lazear is using a one year horizon. That’s a pretty big error, if what he says is right.
It would be interesting to decompose this into the 1999-2007 and 2008-2013 periods separately. All the forecasting agencies have been too optimistic since 2007, and particularly in the recovery. This includes not only the CBO, but also the IMF, Fed, OECD and others, including from the private sector.
The misses have in turn spawned “secular stagnation” as a cottage industry. Clearly, all the above forecasts for the recovery should have incorporated R&R, since “TTisD” was published in 2009. (On the other hand, it is not at all clear to me that R&R’s perspective was formally incorporated into macro models. But do we actually know what R&R think is the right metric? Total debt to GDP? Private debt? HH debt? Corporate? Financial? Govt? Stocks (asset to liability)? Or is it flows (debt service as percent of disposable income)? Is it nominal levels that matter, or as a percent of GDP?) In any event, we have a persistent gap between actual and potential GDP, and given that the private sector has largely deleveraged in the OECD, R&R has fallen out of favor, leading to the cornucopia of alternative explanations for underperforming economies. (It should be noted, however, that this underperformance is global, with the exceptions of Korea, Israel and some of the oil exporters (hint, hint).)
Of these, the most convincing comes from the IMF’s WEO, Chapter 4, which discusses current account adjustments. This is compelling for the weak Euro Zone and EU countries (PIGS, Hungary) which experienced current account crises and massive readjustments in the wake of the financial crisis. See here for the graph: http://www.prienga.com/blog/2014/10/28/oecd-current-accounts This explanation actually combines well with R&R, as it is plausible that the sovereign debt crises of the PIGS were central in the more severe recessions seen there.
Nevertheless, sovereign debt crises in the PIGS do not really explain underperformance in, say, China. Consequently, we are left with a wider forecasting gap with no obvious explanation.
Well, I’m working on one.
given that the private sector has largely deleveraged in the OECD, R&R has fallen out of favor
R&R have fallen out of favor as an explanation for slower than usual recovery? Could you provide a link or two that support that?
One could presume that as George W. Bush’s chairman of the Council of Economic Advisers, Ed Lazear found it be politically inconvenient to speak the truth about a recession heading into the 2008 elections. In Peggy Noonan’s famous words “Is is irresponsible to speculate? It would be irresponsible not to.”
Can we add the Hoover Institution to the pantheon of conservative hackdom along with the American Enterprise Institute and the Heritage Foundation? I know that they are adamant about being called the Hoover Institution at Stanford, in an attempt to co-opt some measure of respectability, to the horror of the Stanford faculty that want nothing to do with them.
Joseph,
What does that have to do with the tea in China?
One can find the WSJ historical forecast data here: http://projects.wsj.com/econforecast/#ind=gdp&r=20&e=55
If you download the historical excel file for that May 2008 forecast, you’ll find that most of the same economists who were forecasting negative growth in Q2 of 2008 were, *in the same survey* forecasting strongly positive growth in Q3 of 2008. The average forecast was 0.2 percent growth for Q2 and 1.8 percent growth for Q3. By my count, only four (of 55) surveyed were forecasting negative GDP growth for Q3. Only 2 (of 55) were forecasting negative growth for the year. The average Q4 growth in that May forecast was 1.6 percent.
That’s not relevant to the WSJ 2008 May forecast?
What a great resource, Viv. I had no idea the WSJ had downloadable forecasts.
That was a thanks, by the way.
Steve Kopits: It is a great resource. I have linked to it about 30-40 times in various posts over the years…
Hey, I don’t see them all. Also, I’m more involved with the topic in this chapter of my book.
But look on the bright side. I read all of WEO Chapter 4. Which I thought well worth reading and a real contribution to the whole secular stagnation debate.
Vivian Darkbloom: Relevant, but not as relevant as the statistics I’ve added in an update.
I’ve said it before and I’ll say it again. Lazear’s comment that “The data are pretty clear that we are not in a recession” was completely reasonable at the time.
Lazear was not denying that there had been a couple of quarters of weak economic growth. But he was making the point that given the characteristics of the weak growth, he did not think they met the technical definitions necessary for the NBER to call a recession. But very importantly, he qualified that view with the following observation in that same interview:
“I would be very surprised if the NBER, looking back at this period, would date this as a recession. Unless, of course, things change in the future and become much more negative. Were they to become more negative, it’s conceivable that they could reach back and date the recession as having started here. But we don’t anticipate that.”
Now, if Lazear were out in left field in not anticipating further negative conditions, you might fault him. But his view was completely consistent with the consensus of economists at the time. As Vivian already pointed out, almost all economists in the WSJ Survey in May of 2008 expected positive quarterly growth in the latter quarters of 2008 and the first few quarters of 2009, just as Lazear did. Moreover, Lazear’s view was confirmed by the May 2008 release of the Survey of Professional Forecasters, which summarized the consensus view as “The forecasters do not expect a contraction in real GDP in any of the next five quarters, as the table on the next page shows.”
Notice also that Lazear in the May 2008 WSJ interview expected that the second quarter of 2008 would be flat. But if you average the second quarter forecasts of the WSJ economists in the May 2008 Survey, you get 0.2%, which is flat. So, Lazear was completely within the consensus.
If I were to deduct points from Lazear, it would not be because of his perfectly reasonable view. Rather, I’d deduct points because his more sanguine vision of future economic activity seems to have been at least partially predicated on the Bush stimulus program having a positive effect. Unfortunately, Lazear shares Menzie’s fantasy that these stimulus programs are effective. But stimulus didn’t work out any better for the Bush Administration than it did for the Obama Adminstration.
“… But we don’t anticipate that.”
Rick, this is the quote that is damning to lazear. he tried to qualify his statement (ie cover his butt) but this clearly states what his view on the future held. he was not anticipating the negative in any realistic form, powered by the politics of the day i would imagine.
Rick Stryker: Wonder who wrote this hysterical bit of prognostication:
Oh, it was you!
Anyway, see the addendum. You can keep on denying Figure 2 all you want, but most reasonable people would say that the data are not “clear”. I’ll just lump you in the set of non-reasonable types.
Keep ’em coming, Rick. You’re a laugh riot!
Menzie,
I said if Romney and his advisors can stay focused. Unfortunately, Romney and his advisers did not stay focused. They didn’t hit hard enough on the poor economic performance. And, inexplicably, they did not exploit the unpopularity of Obamacare. The Republican GOTV was a disaster on election day whereas the Democrats’ GOTV was very effective.
But I’m looking forward to November.
or perhaps romney and the republican party lacked the organizational and leadership skills needed to win that election?
Actually, I think Obama’s fate does resemble that of Carter. Both had the misfortune to govern during oil shocks, and both faced stagnating economies. Certainly the Democratic candidates are keeping well clear of Obama during this election.
If you want to see their respective fates, it’s all there in one graph. Slide 31 of my Columbia presentation.
http://energypolicy.columbia.edu/sites/default/files/energy/Kopits%20-%20Oil%20and%20Economic%20Growth%20%28SIPA%2C%202014%29%20-%20Presentation%20Version%5B1%5D.pdf
If shale oil holds together, Obama could actually have a pretty good last two years and go down in the win column, improbable as that may seem just now.
Menzie, your response to me was very “robust.”
Thanks for the cite Menzie! Joutz, Stekler, and I have another paper that’s particularly relevant once you’re talking about turning points: “Can the Fed Predict the State of the Economy?” Economics Letters 108 (2010), pp. 28-32. We show that the Fed, consistently found to be one of the best forecasters out there, knows the state of the economy for the current quarter, but cannot predict it one-quarter ahead. The performance of forecasters was also recently discussed in “Why Experts Missed the Recession” by Jeff Madrick in the September 24, 2014 issue of the New York Review of Books.
I take it all back, Menzie;
http://taxfoundation.org/article/2015-state-business-tax-climate-index
You are living in a hell hole. I’d consider emigrating if I were you. Maybe to Kansas.
Patrick R. Sullivan: I am still waiting to hear you admit you were in error regarding depth of the downturn in Canada vs. US during the Great Depression. As you recall, you stated unequivocally:
And this statement is wrong.
By the way, the other takeaway from this post is that our forecasting tools are inadequate.
Oh my. I see that Rick Stryker continues his struggle with the English language. Most people who understand English in context also understand the difference between saying “The data are pretty clear that we are not in a recession” and “There is no clear evidence that we are in a recession.” If Lazear had said the latter, then he would have been on safe ground. But that’s not what he said. He said the former, and that is a much stronger statement. Unfortunately, I believe that Rick Stryker quite sincerely does not understand the difference between those two statements. I can only explain his apparent difficulties in understanding contextual English as a manifestation of Asperger’s Syndrome. In any event, Lazear was suggesting that it was highly unlikely we were in a recession. And I’m sorry, but throwaway boilerplate lines like “…of course, things change in the future and become much more negative” does not overcome his more positive and bullish comment. It reminds me of the weatherman who tells us that the last week has been sunny and warm and it’s pretty clear that we aren’t in a rainstorm…but then tries to cover his ass with reminders that it’s always wise to bring an umbrella even if it’s sunny and there’s no reason to think the weather will change. Or again, think of the financial advisor who puffs up the prospects for some investment, and then in a hushed voice notes that past performance is no guarantee of future returns. Lazear was speaking the language of scoundrels and financial analysts…but then I repeat myself.
But I’m looking forward to November.
Why? As Sean Wilentz pointed out the other day, this election is all about the Old Confederacy. We should be mourning the likely outcome. A Confederate victory would be shameful. I blame an aborted Reconstruction. At the time our nation needed Lenin and we got Andrew Johnson instead. And if your reasons for celebrating what the polls are predicting aren’t out of some sentimental attachment to the Lost Cause, then it must be because you see an opportunity to hoodwink clueless voters. When you see a voter you see a sheep to be fleeced. A rube in the big city whose pocket invites picking. People who don’t understand that the Koch brothers are manipulating them are people to be pitied, not taken advantage of. When people read your posts they don’t always understand that you’re simply trying to fool them into voting against their class interests and for your personal interests. Voters are myopic nitwits. It doesn’t help when politicians try to take advantage of that ignorance. For example, when Mitch McConnell tells voters that “kynect” is okay but Obamacare is an abomination, a lot of voters don’t know that Mitch is just lying to them. Why would you look forward to an election that may well bring liars like McConnell into come to power?
2slugs,
Why am I looking forward to November? Because the Washington Post puts the chance of a Republican takeover of the Senate at 93%, the NYT at 70%, and Nate Silver at 63%. I like those odds. Chances are excellent that the Republicans will fully control the Legislative process.
I must say I’m worried about Wisconsin though, with the Gubenatorial race so uncomfortably close. Fortunately, Wisconsin voters are finally getting the information they need to make an informed decision. Better late than never.
Menzie,
I’ve been busy and never got around to answering your Figure 2 point. Let’s take a look at the monthly employment report, which is a key statistic the NBER uses to date business cycles. What did Lazear know at the time? Here’s what the monthly employment report would have looked like:
2008:01 -76 (third estimate)
2008:02 -83 (third estimate)
2008:03 -88 (third estimate)
2008:04 -28 (second estimate)
2008:05 -49 (first estimate)
sum = -324
Looking back on the 2001 recession which started in March 2001, Lazear would know these numbers:
2001-04-01 -281
2001-05-01 -38
2001-06-01 -129
2001-07-01 -115
2001-08-01 -158
sum = -721
2001 was an extremely tepid recession and yet the comparable employment loss was just half of the 2001 period. It looks like a slowdown for sure, but it doesn’t rise to the level of the 2001 recession. Now of course later on those initial 2008 numbers got revised to
2008-01-01 15
2008-02-01 -86
2008-03-01 -80
2008-04-01 -214
2008-05-01 -182
sum = -547
Of course, if Lazear could have foreseen the revisions coming, then he’d have likely said that the economy was in a recession by May of 2008, starting at the beginning of the year approximately. But that’s why the question of what future economic activity will be is crucial. As Lazear correctly noted, if future growth comes in much weaker, then the initial numbers are likely to be revised down substantially and the NBER could well date it as a recession. But Lazear didn’t expect at the time for future economic activity to be weak, consistent with the broad consensus of the economic forecasters at that time, and thus did not expect those broad downward revisions.
Lazear had a reasonable point given what he knew at the time.
Rick Stryker: Yes, I understand being busy. I recall one comment where you said you’d be too busy to do any more commenting, and another that said you’d had it commenting on this blog (December 2013). Apparently the work load decreased.
I was on CEA staff in June 2000-June 2001. We saw plenty of data. Some of it worrisome, but in general ambiguous. I don’t think anybody at the member level ever said publicly “The data are pretty clear that we are not in a recession.”
Rick Stryker: Maybe you should read what Jeffrey Frankel wrote in 2008. Since he is on the NBER Business Cycle Dating Committee, I suspect he has better feeling on what is reasonable than either of us (unless you too are on the NBER BCDC — something I sincerely doubt).
The WSJ forecast link via Vivian is very valuable.
I wonder, did so far anybody analyze the data with respect to
a) when people changed their forecasts,
b) did they just account for the changes already happened, or extrapolate, and
c) what the time constants implicit were for the recovery predictions
links, hints very welcome
Rick,
You have to understand that Menzie will criticize your posts but not take any position you can call his own. I only know of two positions he has taken. First, he believes there should be no limit on SNAP payments – let’s sign up the whole world! Second, we now know that he would rather have a “robust” analysis than an accurate analysis. Well, there a lot of us who like Sudoku and Menzie has taken it to an art form.
Rick Stryker FYI. JDH’s emoticon went from a neutral in Oct 2007 to a frown in Jan 2008. Perhaps Lazear should have checked out this blog before making his rather upbeat prognosis in May 2008.
Still not sure if you’re voting Republican because you’re really a closet Confederate or if you just cynically believe in manipulating the dim witted who watch Fox News. Oh well, if the polls turn out to be right at least I’ll be able to enjoy the comedy of GOP congress critters talking about “nullification” and how Adam & Eve rode around on the backs of dinosaurs. And yes…there is one GOP Senate candidate who is narrowly leading in a tight race who campaigned on exactly those issues.
2slugs,
My personal emoticon at the time also went to a frown. At the time, I also thought Lazear would turn out to be wrong. But so what? My point has been that in real time it’s never clear one way or the other what the situation is. I think given the data at the time Lazear’s view that we had not fallen into recession was a reasonable point of view, even if I didn’t really agree with that view at the time. Lazear took a clear stand without all the caveating that people often do. His view turned out to be wrong. That doesn’t discredit him or invalidate his views in general. In fact I think he deserves credit for being willing to take an unambiguous position, as I’ve suggested before.
Not sure what your consistent references to the Confederacy are about.
Rick Stryker: Ok, so let’s see. On one side Jeffrey Frankel and Martin Feldstein comment, indicating activity could be plausibly interpreted as declining. Frankel is a NBER Business Cycle Dating Committee member, while Feldstein was the Chair of the NBER BCDC. Jim Hamilton’s indicator is frowning as well. On the other side is Lazear and loyal defender Rick Stryker. Whose arguments to give greater weight to…?
stryker on lazear being wrong
“But so what?”
because he is the mouthpiece to which president bush was taking direction on economic policy. and he was wrong. just like we had mouthpieces on weapons of mass destruction in iraq-and they were wrong.
so your comment “But so what?” suggests you really do not care whether our leaders get it right or wrong. or you only care depending upon which party is in office. but there were millions of people in this country who paid a significant price for lazear and company to be wrong. did any of those decision makers pay a price for their mistake? no.
“His view turned out to be wrong. That doesn’t discredit him or invalidate his views in general. In fact I think he deserves credit for being willing to take an unambiguous position” he was paid to be correct in his views. being confident in your convictions when you are wrong does not deserve credit. why does he deserve special status for being wrong? he cannot be defended except by another ideologue.