That’s the name of a new NBER working paper, and a paper presented at ASSA (slides; I didn’t get to see myself). As I get to teach applied econometrics for public policy this semester, I thought this was interesting paper. From the abstract:
Most empirical papers in economics focus on two aspects of their results: whether the estimates are statistically significantly different from zero and the interpretation of the point estimates. This focus obscures important information about the implications of the results for economically interesting hypotheses about values of the parameters other than zero, and in some cases, about the strength of the evidence against values of zero. This limitation can be overcome by reporting confidence intervals for papers’ main estimates and discussing their economic interpretation.
This short paper is particularly relevant, given the debate over “p-hacking” (which was ongoing when I last taught this class — I think I referred to this paper). Specifically:
…a common approach to discussing empirical results often leaves out important information about the results’ implications. At a general level, addressing this omission is straightforward. What is need is information about the implications of the results for hypotheses of interest. And although there are various ways of providing this information, a natural one is to report and discuss a confidence interval. In contrast to reporting a point estimate and whether it is statistically significantly different from zero, reporting a confidence interval provides information about the full range of possible values of the parameter.
One example in the paper, macroeconomically oriented:
…the fiscal multiplier—the short-run output effect of a one-unit increase in government purchases with the economy in a specific set of circumstances (for example, with output below normal and with a particular assumption about the behavior of monetary policy). There are some models where the multiplier is zero (notably, flexible price models with inelastic supply), so here the null hypothesis of zero is an interesting one. But other values are also potentially important. One focal value is a multiplier of one, which is both the value predicted by some models under certain conditions (Woodford 2011) and the boundary between stimulus increasing or decreasing private economic activity. A policymaker designing a stimulus package might be interested in comparisons with results from prior work about multipliers for various types of tax cuts. And, as with the return to education, values obtained in previous work, such as the figure of 1.8 suggested in a recent survey of cross-sectional research by Chodorow-Reich (2019), are also of interest. Again, a focus on the point estimate and whether it is statistically significantly different from zero is misplaced if a reader’s interest is in knowing what the evidence tells us about any of these various other possible values.
Still, it’s important to remember what confidence interval is, as there is much confusion on this issue. It is an interval that:
“Were this procedure to be repeated on numerous samples, the fraction of calculated confidence intervals (which would differ for each sample) that encompass the true population parameter would tend toward 95%.”