Forecasts Then and Now: A Cautionary Tale

The White House released its midterm forecast on Tuesday. Some thoughts on forecasts around turning points.


The announcement can be found here, with CEA Chair Lazear’s discussion here. The forecast is summarized in this table:


ceatab1.jpg


From the press conference:


CHAIRMAN LAZEAR: Thanks a lot. I appreciate you all joining us on this call. Today we’re releasing the administration’s economic forecast that will be used for the President’s fiscal year 2008 budget.


These forecasts are developed semi-annually by the Council of Economic Advisers, the Treasury Department, and the Office of Management and Budget. And we do these together.


The updated forecast is similar to private sector forecasts, and it’s also similar to the administration’s past forecast. I should point out that this is a routine process. As I mentioned earlier, it’s done twice a year, and it is done by a professional staff. These are career employees who tend to be long-term individuals, who have been in the government for a while, and they are not specific to any one or the other administration.


Let me give you some of the details that are associated with this particular forecast. We are projecting real GDP to grow at 3.1 percent in 2006, and that’s on a Q4, quarter four to quarter four basis. And we’re projecting 2.9 percent for 2007. These growth rates are similar to the U.S. historic average. And they are also similar to the ones that the market is giving us in terms of the blue chip forecast.

To investigate further the implications, I’ve plotted log real GDP in 2000 billions of Chained dollars (blue), the CEA forecast (red squares), the forecast from the Society of Professional Forecasters (black), and the Deutsche Bank forecast (green).


ceafcast.gif

Figure 1: Log real GDP (October 27 release, blue line), CEA forecast (11/21, red squares), SPF forecast (11/13, black line), and Deustch Bank forecast (11/17, green line). Sources: BEA, CEA, Philadelphia Fed, and DB.

The graph confirms the high level of agreement between the CEA and consensus forecasts. Doubts are raised by several observers, inlcuding by Calculated Risk, but these doubts apply to many of the extant forecasts. The issue is that these types of forecasts can go grieviously wrong around turning points. In this context, I thought it would be of some use to see how the consensus forecast performed around the last turning point. In Figure 2 below, I plot the 1/31 vintage of log GDP in 1996 billions of Chained dollars (blue), the Society of Professional Forecasters forecast of 2/10 (red), and the last vintage (11/25/03) of the GDP data available in 1996 Chained dollars (green).


lastrecession.gif

Figure 2: Log real GDP (1/31/01 release), SPF forecast (2/10/01), and log real GDP (11/25/03 release). NBER recession dates in gray. Sources: BEA via St. Louis Fed ALFRED, Philadelphia Fed and NBER.

What’s interesting is that even on the eve of the recession, the mean forecast was for resumed growth in 2001q2. However, the actual revised data that came out over two and a half years afterward (the November 2003 vintage) indicates that, not only was the level of real GDP in 2000q4 1.7 ppts lower than that perceived at the time, but that output was actually decreasing in 2001q1, rather than rising by 0.8 ppts (annualized) as forecasters thought in February of 2001. Whether you believe or disbelieve the consensus forecasts thus depends upon whether you think we’re near a turning point or not. On that, reasonable people can disagree.

Technorati Tags: ,
, and
.

10 thoughts on “Forecasts Then and Now: A Cautionary Tale

  1. Economic Investigations

    Forecasting Performance

    Menzie Chinn at Econbrowser (which is slowly becoming one of my favorite if not my most favorite blog) takes at look at forecasting performance, whether done by the government or by private forecasters: Forecasts Then and Now: A Cautionary Tale The con…

  2. C Thomson

    Older readers may remember that Denis Healey, when Chancellor of the UK Exchequer, said that he wanted to do for economic forecasters what the Boston Strangler had done for door to door salesmen.
    Economics is a powerful tool for explaining what has already happened. As to forecasting the future…well, there is always the consensus or just plain naive extrapolation.

  3. claus vistesen

    Hi Mr. Chinn,
    I have to say that I am no expert on this but the ECB research department had a paper on something related recently, that is; the forecasting accuracy of Fed Watchers.
    http://www.ecb.int/pub/pdf/scpwps/ecbwp695.pdf
    From the Abstract;
    _’The paper shows that there is a substantial degree of heterogeneity in forecast accuracy
    among Fed watchers. Based on a novel database for 268 professional forecasters since
    1999, the average forecast error of FOMC decisions varies 5 to 10 basis points
    between the best and worst-performers across the sample. This heterogeneity is found
    to be related to both the skills of analysts such as their educational and employment
    backgrounds and to geography.’_

  4. calmo

    This reminds me of Fisher’s “bad data” only determined much much later:

    However, the actual revised data that came out over two and a half years afterward (the November 2003 vintage) indicates that, not only was the level of real GDP in 2000q4 1.7 ppts lower than that perceived at the time, but that output was actually decreasing in 2001q1, rather than rising by 0.8 ppts (annualized) as forecasters thought in February of 2001

    and it is hard to swallow that it took that long to see that it was that far off. It does look like a genuine mislead, not exactly a Chamberline, but asking alot of the reasonable people to accept the misdirection as merely ‘bad data’.
    Reasonable people can disagree only so far, yes? Could be reasonable people count on better performance than this from authorities. [The amount of time Lazear spends on legitimizing the report’s authority is self-molesting, yes?]
    Were it not for the turns, reasonable people (who are famously momentum bound and unturnable) could dispense with the authorities who never detect these turns in a timely manner unless its with crys of “the worst is over”.
    Or is it worse? They know the record for housing turns and do not disclose this knowledge fearing that it might exacerbate present trends?
    But some reasonable people do know and do not agree with that “worst is over” assessment and want to distinguish themselves from those that merely want to market the administration’s “growing economy”.

  5. algernon

    calmo,
    In the past, you’ve disagreed with my generalization that it is in the nature of gov’t to lie. Here you are saying they are probably lying, & ironically I feel compelled to defend them by saying you may be over-estimating the knowledge the bureaucracy possesses.
    Much of the lack of truth that issues from the gov’t is not conspiratorial. Measuring the economy is not a piece of cake.

  6. calmo

    algernon, how could I possibly disagree with your generalizations of the past? Surely, there has been some mistake…of grave proportions.
    I B wounded.
    Here you are saying they are probably lying..
    where could I possibly be saying that? Surely you jest. How could I possibly say “lying” when I printed “mislead”, “misdirecting”?
    But this bit distracts me…possibly my total disarmament:
    ironically I feel compelled to defend them by saying you may be over-estimating the knowledge the bureaucracy possesses
    Ordinarily I’d surrender here, but I’ve had a few…and don’t know when I’ve been smoted.
    And I feel compelled to join you (and hordes of underemployed (but of course still reasonable) voices in a Wagnerian-scale chorus)
    proclaiming the heavy burden that weighs on those entrusted to discover this economic knowledge, or failing that, to deliver an Intelligence Estimate of same, or failing that, to use a cheery reading from some other more prosperous place and time.
    I have decided this question, ‘Incompetent or mendacious?’ is itself mendacious.
    Ok, long after many others because compared to them, I am incompetent.

  7. menzie chinn

    Claus Vistesen: Thanks for the reference. I had a discussion with one of the authors about the findings back in June. They are quite surprising.

    calmo and Algernon: I think it is useful to distinguish between the (i) forecast, which is produced by the professional staff of the Troika of CEA, OMB and Treasury, and (ii) the spin that is applied to the figures in the forecast. Having had some small opportunity to observe how the forecast is produced, I have faith that these reported figures are not driven by political imperatives. The surrounding verbal discussion of the forecast I leave to readers to assess. The policy-level officials clearly have an incentive to interpret the forecasts in the best possible light, which might — or might not — coincide with the views of the forecasters.

    By the way, this issue highlights why it is important to defend the independence of the career professional staff in various scientific and technical areas — something that has been under assault by some policy-level appointees over the past six years in various arenas.

  8. calmo

    Thanks Menzie, maybe I was spurred on by Lazear’s intro:
    Today we’re releasing the administration’s economic forecast that will be used for the President’s fiscal year 2008 budget.
    These forecasts are developed semi-annually by the Council of Economic Advisers, the Treasury Department, and the Office of Management and Budget. And we do these together.
    The updated forecast is similar to private sector forecasts, and it’s also similar to the administration’s past forecast. I should point out that this is a routine process. As I mentioned earlier, it’s done twice a year, and it is done by a professional staff. These are career employees who tend to be long-term individuals, who have been in the government for a while, and they are not specific to any one or the other administration.

    and his efforts to include the long-term professionals (and not draw attention to his predecessor) who have more ‘independence’ from the political interests. [Must be a reason he’s warning us, yes?]
    It’s sorta like hearing from a former professional in that bureaucracy informing us that times have changed. 8)

  9. Sonia

    Techniques for forecasting a recession are speculative. Any projection that deviated in obvious ways strongly from the consensus would risk being radically wrong. Also a forecast strongly outside the consensus in either direction would be suspect as supporting a political agenda (e.g. more tax cuts or more spending). Furthermore a forecast of recession by any administration could be selfulfilling.
    Discussions by the political appointees around the forecast are likely to sin by omission rather than commission. The interesting comparisons of forecasts and projections are often in the allocation of income shares and in the relationships of current law spending and revenue projections to income shares.
    Menzie Chinn makes an important point re maintaining the independence of the professional staff. A key question also is whether the professional staff are asked all the questions or just the questions likely to yield a desired answer.

  10. menzie chinn

    Sonia: I agree. The devil is in the details. Or more specifically, the impact on Federal revenues and outlays is driven by the assumptions regarding the components of GDP and national income.

Comments are closed.