I’ve just completed a new research paper with University of Chicago Professor Cynthia Wu on the Effects of Index-Fund Investing on Commodity Futures Prices. Here was our motivation for writing the paper:
The last decade has seen a phenomenal increased participation by financial investors in commodity futures markets. A typical strategy is to take a long position in a near futures contract, and as the contract nears maturity, sell the position and assume a new long position in the next contract, with the goal being to create an artificial asset that tracks price changes in the underlying commodity. Barclays Capital estimated that exchange traded financial products following such strategies grew from negligible amounts in 2003 to a quarter trillion dollars by 2008 (Irwin and Sanders (2011)). Stoll and Whaley (2010) found that in recent years up to half of the open interest in outstanding agricultural commodity futures contracts was held by institutions characterized by the Commodity Futures Trading Commission (CFTC) as commodity index traders.
Hedge fund manager Michael Masters argued in testimony before the U.S. Senate in 2008 that purchases of commodity futures contracts by index funds must have influenced prices:
Index Speculator demand is distinctly different from Traditional Speculator demand; it
arises purely from portfolio allocation decisions. When an Institutional Investor decides
to allocate 2% to commodities futures, for example, they come to the market with a set
amount of money. They are not concerned with the price per unit; they will buy as many
futures contracts as they need, at whatever price is necessary, until all of their money
has been “put to work.” Their insensitivity to price multiplies their impact on commodity
markets.
In testimony the following year before the CFTC, Masters claimed:
Buying pressure from Index
Speculators overwhelmed selling pressure from producers and the result
was skyrocketing commodity prices.
In our new paper, Cynthia Wu and I review a simple model of how such market pressure might influence commodity futures prices. The basic idea is that the more futures contracts the funds want to hold, the more risk the counterparties who short the contract are exposed to. According to the model, the futures price must be bid high enough to compensate the short side for absorbing the risk. This compensation comes in the form of an expected profit to the short side of the futures contract. The empirical implication of the model is that if the long position of index funds is indeed exerting an effect on commodity futures prices, then the notional value of index-fund futures contract holdings should help predict the returns over time on the contracts themselves.
The framework thus offers an interpretation for a large previous literature that has looked for evidence of predictability of commodity prices based on various measures related to index-fund investing. Here is a brief summary of those previous studies provided by our paper:
Brunetti, Buyuksahin, and Harris (2011) used proprietary
CFTC data over 2005-2009 on daily positions of traders disaggregated into merchants,
manufacturers, floor brokers, swap dealers, and hedge funds. They found that changes in net
positions of any of the groups could not help to predict changes in the prices of futures contracts
for the three commodities they studied (crude oil, natural gas, and corn). Sanders and Irwin
(2011a) used the CFTC’s publicly available Disaggregated Commitment of Traders Report on
weekly net positions of swap dealers, and found these were no help in predicting returns on
14 different commodity futures contracts over 2006-2009. Sanders and Irwin (2011b) used
proprietary CFTC data to extend the public Supplemental Commitment of Traders (SCOT),
which categorizes certain participants as commodity index traders, back to 2004. They found
that changes in the positions of index traders did not help predict weekly returns for corn
or wheat but found some predictability for soybeans under some specifications. Stoll and Whaley (2010) used the public SCOT for 12 agricultural commodities over 2006-2009 and
found that changes in the long positions of commodity index traders predicted weekly returns
for cotton contracts but none of the other 11 commodities. Alquist and Gervais (2011) used
the public CFTC Commitment of Traders Report to measure net positions of commercial and
non-commercial traders, and found that changes in either category could not predict monthly
changes in oil prices or the futures-spot spread over 2003-2010, though there was statistically
significant predictability when the sample was extended back to 1993. Irwin and Sanders
(2012) used the CFTC’s Index Investment Data on quarterly positions in 19 commodities
held by commodity index funds. They found in a pooled regression that changes in these positions
did not predict futures returns over 2008-2011. They also separately analyzed whether
changes in futures positions of a particular oil- or gas-specific exchange traded fund could
predict daily returns on those contracts over 2006-2011, and again found no predictability.
Buyuksahin and Harris (2011) used proprietary CFTC data on daily positions broken down
by non-commercials, commercials, swap dealers, hedge funds, and floor broker-dealers. They
found the last category could help predict changes in oil futures prices one day ahead, but no
predictability for any of the other categories or other horizons. By contrast, Singleton (2011)
found that a variety of measures, including a 13-week change in index-fund holdings imputed
from the SCOT, could help predict weekly and monthly returns on crude oil futures contracts
over September 2006 to January 2010.
The model we investigated has a specific implication for how index-fund holdings could influence futures prices– the notional holdings of index-fund investors should help predict the returns for the counterparties to those contracts.
For 12 agricultural commodities, since 2006 the CFTC has been providing through its Supplemental Commitments of Traders Report weekly positions for traders it characterizes as “replicating a commodity index by establishing long futures positions in the component markets and then rolling those positions forward from future to future using a fixed methodology”. We looked at a variety of specifications in which the notional holdings associated with those positions might help predict returns on any of the futures contracts. Consistent with the findings of the other researchers summarized above who used different data sets and methods, we found essentially no helpful forecasting relations for any of the 12 agricultural commodities.
This was something of a disappointment for us, since we had what we thought was a neat model to capture the effect that Masters expected to see. But we found no basis in the data for assigning a value other than zero to any of the key parameters of that model– there just is no forecasting relation in the data between index fund positions in these 12 agricultural commodities and subsequent changes in their futures prices.
As noted above, the literature has reported some more mixed results with oil prices as opposed to agricultural prices. I will discuss what we found for those data in a follow-up post.
Model? I thought that it was very simple. You had a huge bear market in commodities for nearly two decades. During that bear market high grading of mines and the closing of plantations ensured that metals and grains prices would go much higher eventually. But as prices were dropping and supply was drying up there was growing demand from a developing world that had liberalized local economies and encouraged the growth of middle classes that consumed more commodities. Eventually the fall in supply and rising demand reversed the price trends and investors began to play catch-up in a world that made it harder to get approval for the type of projects that were needed.
There is no mystery here. Speculation plays a part but a small part unless there is action that manages to crash the financial system and take demand much lower. There are no more oil producers that can drive prices lower by opening up idle capacity. Prices will drop because of painful contractions not new fields bringing on new production. And while we could have supply shocks in zinc, lead and a few other metals it is difficult to see a supply side solution to higher prices for a while.
Professor,
You are doing great research. Keep it up. I look forward to your post on oil.
Please forward results or the paper to Dean Baker and Paul Krugman.
JDH I’m curious…and especially curious that Ricardo chimed in without mentioning it. But is there any particular reason that you and Cynthia didn’t include gold or other rare/precious metals as one of your indices?
Thank you, Professor Hamilton, for using econometrics in place of evidence to support your a priori belief. Amazing how you only seem to reference papers that used some sort of method that you agreed with. please do everyone a favor, and next time be sure to include the fact that you’re using econometrics to support your ideology. You’re a joke.
Vangel, your analysis is devoid of institutional detail and basic facts. Explaining prices based on some long term trend when the evidence doesn’t support it is laughable.
Minneapolis FED; Kocherlakota said the U.S. inflation rate is signaling a similar spike in the maximum employment level in the U.S. and the Fed should “be responsive to such signals.”
Which ‘inflation signal’ would that be, the one that government publishes, or some private data that we have not yet seen?
Payam Sharifi: You mean some people form their conclusions based on ideological belief, regardless of what the evidence shows?
I believe I have cited above every single academic study that has looked for evidence that index-fund positions can predict futures returns. If you know of another, please call it to my attention.
It is true that there are other studies that use different methodologies, and these are discussed in the paper.
Slug,
Help me out here. What does gold have to do with speculation? Are you really that ignorant of the gold indicator?
Imbalances in the future markets coincided with new regs that allowed speculators bigger positions that the traditional 30% of open interest. (the classic grease for the wheels of commerce)
Today speculators can account for more than 80% of open interest resulting in wide swings that go beyond any semblance of market fundamentals. The telling example was the grain markets several years ago where specs ran the price of grain into the stratosphere; based on the knowledge that true hedgers, farmers and grain elevators were required either by law or bank agreements to be 100% hedged. They ran the prices to levels were the cost of margin calls exceeded the value of the commodity being hedged, a classic short squeeze.
Commodity markets will return to equilibrium when speculators are no longer allowed to control the open interest in the futures markets.
This paper deals with prices index and financial participants in the prices index. This paper concludes index commodities traders have no bearings on prices. Index traders purchase a price, but do not contribute to its formation.
When revisiting history, commodities volume output, prices, stocks warehousing and flows, have always been at the core attention of the public administration management.
When commodities stocks were above normal average, the farm board was making direct loans to the cooperatives, the loans amounts
were set at a fixed market prices of the commodities involved. The risk was, that too high of an administrated price would detract buyers from the physical commodities. In order to tamper the scarcity of buyers, the farm board made direct purchases of commodities from the cooperatives. The risk was that higher crops elsewhere would set lower worldwide price for the same commodity (ies) and domestic price be driven down.This was the case in 1930, the shortage of counterparts at a higher fixed price drove the commodities markets on a downwards spiral.
Economic necessity and hence reward is transparent in the price formation, in the stocks management, warehousing, but the index trading is apparently of little added value as it reflects on none of the functions, warehousing, prices , stocks management and stocks flows. The paradox is that index trading may be rewarded through prices fluctuation only. The paradox may be explained if commodities index funds trading are the stoics instruments of a worldwide prices mechanic equalizer.
In this paper the black box linked to the price cybernetic has been left aside, as it deals only with the index price. Inside the box the publication of GAO reveals that participants are less than equal to the random distribution of profits (please read P 20, GAO “Regulators will need more information to fully monitor compliance with regulations when implemented”)
When necessity makes laws, banks must be recapitalized, when shareholders cannot afford to take the losses tied to their misfortune, the central banks may assist. The Value At Risk as an indisputable mathematical and efficient shield against financial mismanagement (subprime derivatives, commodities, mortgages derivatives) remains the most doubtful risk.
Ricardo I was just wondering if you thought the buying of gold index funds predicted the future price of gold. In fact, is there anything that predicts the future price of gold? You’re the resident gold bug here, so presumably you think gold prices do more than just take a random walk.
The real interesting effect of the growth of index funds was on same crop year spread relationships. Before index funds, spreads would seldom approach carrying charges. The massive volume that funds needed to move from the expiring delivery month into the next delivery month, continually drove these spreads to carry. In effect the index funds were paying grain merchants to hold inventory and made it harder for the market to be bid up and draw inventory out of storage. I’m not sure how this affected overall price levels but it greatly changed the way that markets traded.
Slug,
Sometimes you know so much to be so ignorant.
A gold bug is someone who invests in gold believing that it is money making commodity. I am not a gold bug. I do not have one dollar in gold and at various times here I have explained why.
Your question about predictions of the future price of gold once again reveals your ignorance. You are trapped in a world of econometric statistics and formulae when economics is about people. The only way you can know the future price of gold is to get into the minds of the members of the Federal Reserve and know what “magic” they are planning to perform next.
Gold is a leading indicator. I realize this is a difficult concept for you but what that means is that changes in gold indicate what the tendencies are for the future concerning the money supply and the value of the currency. The price of gold is a reaction of the market to the money supply relative to monetary demand. Basic monetary stuff.
So essentially your question is nonsensical to anyone who understands monetary policy.