In my previous post
I described a new research paper with University of Chicago Professor Cynthia Wu on the Effects of Index-Fund Investing on Commodity Futures Prices. Previously I discussed what we found for the prices of agricultural commodities. Here I review our findings about oil prices.

Part of the interest in a possible effect of commodity-index funds on oil prices comes from testimony before the U.S. Senate by hedge fund manager Michael Masters, in which he produced a provocative graph of oil prices against an estimate of the number of crude oil futures contracts held by commodity-index funds. We reproduced his methodology to update his graph below. The figure certainly seems to suggest a strong connection between these two series, particularly during 2008 and 2009.

Price of near crude oil contract (left scale) and number of crude oil contracts held by index traders as imputed by Masters’ method (right scale). Source: Hamilton and Wu (2012).

The CFTC does not release a direct estimate of index-fund holdings of crude oil contracts at the weekly frequency of data plotted above. Masters therefore used an indirect method based on CFTC-reported holdings by commodity-index funds of 3 particular agricultural commodities to construct the green line in the graph above. His reasoning was that most index funds follow one of two popular strategies, trying to track either the S&P-Goldman Sachs Commodity Index or the Dow Jones-UBS (formerly Dow Jones-AIG) Commodity Index. He noted that soybean oil, one of the 12 agricultural commodities for which CFTC does report a weekly estimate of index-fund positions, is included in the Dow Jones but not the Goldman strategy. Masters’ proposal was that, by using the CFTC estimate of holdings of soybean oil by index funds, and from the known weights that the Dow Jones strategy calls for holding soybean oil relative to crude oil, one could infer how many crude oil contracts were being held by funds following the Dow Jones strategy. Similarly, the Goldman strategy includes Kansas City wheat and feeder cattle, whereas the Dow Jones index does not, so either of these commodities (Masters used the average) could generate an implied holding of crude oil contracts by those trying to replicate the Goldman Sachs Commodity Index. Summing these two estimates produced the green line in the figure above. Thus this estimate of crude oil holdings is actually based on reported holdings in soybean oil, Kansas City wheat, and feeder cattle contracts.

This approach of imputing crude oil holdings from a few agricultural commodities has been sharply criticized by Irwin and Sanders (2011), who note that the implied estimates using Kansas City wheat often differ significantly from those based on feeder cattle. They also document that, for dates for which we have separate reasonable estimates of crude oil contracts held by commodity-index funds, the actual holdings differ substantially from the series plotted in the green line above. (See my discussion of Irwin and Sanders’ paper in Econbrowser last August).

In my new paper with Cynthia Wu, we demonstrate that Masters’ idea for imputing crude oil holdings from agricultural measures does not require finding a commodity that appears in one index but not the other. Algebraically, the method can be thought of as simply solving a system of two equations to determine two unknowns. In fact under Masters’ assumption, it would be possible to infer crude oil holdings from almost any two arbitrary agricultural commodities. The graph below shows what those inferences look like if one uses soybean oil plus any one of the indicated second commodities. The inferred series for crude oil holdings is quite sensitive to which agricultural series one uses. The figure also plots a regression method that we developed that makes use of all 12 commodities together, which can be viewed as a generalization of Masters’ averaging idea.

Holdings of crude oil contracts held by commodity-index traders imputed by (a) soybean oil and one other agricultural commodity; (b) Masters’ method; (c) regression method. Source: Hamilton and Wu (2012).

A recent paper by Stanford Professor Ken Singleton found that the 13-week change in Masters’ estimates of crude oil holdings can help forecast returns on crude oil futures contracts. Cynthia and I were able to reproduce this finding, though the level and 1-week change of the Masters series don’t have any predictive power.

Returning to the first figure above, the striking feature of the Masters series is that it collapses as the recession worsened in 2008 but began to rebound sharply before the recovery began, features that help give it apparent predictive power over the sample period that Singleton originally studied. Since Singleton’s paper was written, we now have 2 more years of data with which to see if the relation has true predictive power. We estimated the forecasting regression using a sample that ended at the same date as Singleton’s original analysis (January 12, 2010), and then used those regression estimates to try to forecast crude oil futures prices over January 17, 2010 to January 3, 2012. We found that in this out-of-sample exercise, the regression actually did 22% worse than one would have done if one simply used the naive forecast that futures prices would never change.

Interestingly, we found that Masters’ measure not only appeared to forecast changes in crude oil prices over the 2006-2009 period, but would equally appear to have been able to predict changes in the S&P500 stock price index over that period. However, this relation, too, turns out to perform worse out-of-sample than the naive prediction that stock prices will never change.

Our conclusion is that the correlation between 13-week changes in commodity-index holdings of agricultural futures contracts over 2006-2009 and other series such as changes in crude oil or stock prices is likely to just be a coincidence. Overall, we find very little support in the data for the claim that index buying exerted significant effect on commodity futures prices.