Felix Salmon

How commodity indices broke the wheat futures market

By Felix Salmon
June 24, 2009

Back in March 2008, Diana Henriques noted something very odd: a large number of futures contracts traded in Chicago were expiring at levels much higher than the spot cash price. She said at the time that “economists who have been studying this phenomenon say they are at a loss to explain it”.

This very odd phenomenon — which caused farmers a lot of harm — has now been explained in a 247-page report from the Senate investigations subcommittee, entitled Excessive Speculation in the Wheat Market. The main PDF is here; there are exhibits and addenda here. The culprit, it turns out, is index traders.

The rise in the basis between the futures price and the cash price is a function of the rise of commodity indices, and investors buying a basket of commodities. This affected the wheat market particularly badly, as explained in footnote 213 of the report, partly because of the ease of storing wheat:

Aside from wheat, the other commodity markets in which index traders hold a substantial share of the long open interest are the futures markets for two livestock commodities, lean hogs and live cattle. Lean hog futures contracts are financially settled, meaning that the price of the expiring futures contract is set at the price of the commodity in the cash market at contract expiration. By definition, therefore, lean hog futures and cash prices will be equal at settlement, so there is no problem with convergence. Live cattle, unlike grain, cannot be placed in storage from one contract expiration to another. That constraint means there is always an active cash market for live cattle at contract expiration that helps to force convergence.

The Senate subcommittee recommends that the futures exchanges should curb speculation in the futures market in order to bring the basis between futures and cash back down to a reasonable level. It’s coming down already, but it’s still extremely high, at over a dollar a bushel:


This whole thing reminds me slightly of the way in which the USO oil ETF helped to exacerbate contango in the oil market. And in general, it seems that attempts by investors and banks to construct financial instruments which give simple exposure to commodities have not worked very well. Chalk up another failed financial innovation.

15 comments so far | RSS Comments RSS


As you often say, this report’s conclusion doesn’t pass the smell test.

The first troubling aspect is that there is absolutely no attempt at statistical analysis. The assertion that index trading has been responsible for the rise in the basis is apparently based on the testimony of market participants (who may be biased), and on casual empiricism using graphs such as ES-1, ES-2, and ES-3, which show a general increase in index trader participation and a general increase in the basis over the last several years. If you actually look at the graphs, though, it’s far from clear that the trends have anything to do with one another. The sharpest increases in the basis, for example, come in 2008, a period when index trader participation in the market was flat and then down. There is no clear evidence even of correlation, let alone causation.

The second troubling aspect is that by definition, index traders should have no impact on the level of the basis at settlement. As the report itself confirms, index traders generally hold contracts beyond the front month, and when those become front-month contracts they are sold in favor of farther-dated futures. Thus index traders should never be holding futures contracts at expiry, and should not have an impact on the basis.

Relatedly, the report focuses on the daily (pre-expiry) difference between futures and cash prices, implying that an increase in this measure should be seen as problematic. But in general there is no reason to expect that the futures price should track the cash price at all prior to contract expiry, since it is determined by expected future prices and risk premia.

Third, the report itself hints at another market factor that is a clear candidate for explaining the widening basis, without apparently recognizing it as such. On p. 35, the report mentions that “For grain futures contracts, the rules of the exchanges specify a narrow category of sellers who may make an actual delivery of grain pursuant to those contracts.” For a deliverable commodity, the convergence of the cash and futures prices at contract expiry is reliant upon the ability of market participants to arbitrage the basis through physical delivery. If a very small number of players are essentially granted oligopoly power over delivery, then the arbitrage will clearly only be executed in cases where it is financially beneficial to those players. Incidentally, switching to a cash-settled futures contract would solve the basis problem by definition.

“Speculators” are an easy target for blame in general, so it’s important to be wary of reports pointing fingers at them. I’m usually very impressed by the level of analysis and skepticism on this blog, so I was disappointed to see an uncritical presentation of this report as having identified “the culprit.”

Posted by Nick | Report as abusive


The report relies on basic supply and demand analysis. The authors note that there was a huge increase in demand for long futures contracts (index traders) without a similar increase in supply of long futures contracts (which is by definition equal to demand for short futures contracts). The only way for the market to reach equilibrium was for the price to rise.

You argue: “by definition, index traders should have no impact on the level of the basis at settlement.” You seem to be ruling out by assumption the possibility that an increase in futures prices can drive up the expectations of both buyers and sellers regarding future prices in the cash market and thus affect current and future cash prices of the product. But what justification do you offer for your assumptions?

The report indicates that it is precisely because of these complex price dynamics (and the storability of wheat) that index traders affect the market: “Thus, it is the speculators who make the inventory decisions. Inventory is accumulated when speculators are willing to pay enough more for distant contracts than nearby to encourage accumulation and hedging, and inventory is liquidated when speculators will pay little more or even less for the more distant contracts.”

How can any regression analysis on prices address this issue? The only time series we have is the one that was realized. It can give us no information whatsoever on what prices would have been if the participants in the market had been different.



“The only time series we have is the one that was realized. It can give us no information whatsoever on what prices would have been if the participants in the market had been different.”

So the whole report is pure speculation? And this is your defense of it?

Posted by bzimmy | Report as abusive


Do you actually think that data analysis can answer the question of what economic model correctly portrays the real economy? If so, I don’t think you’ve been exposed to econometrics. The general conclusion from economic data analysis is that there is a vast range of models that are all consistent with the data.

Yes, you can go ahead and make bad assumptions — that happen not to contradict the data — but that won’t turn it into good quality economic analysis. (Though it might help you price a CDO.)



Glad to hear we agree that the data may be consistent with many explanations and therefore writing a post (or govt report) titled: “How commodity indices broke the futures market” would not be prudent. Don’t think Nick is disagreeing with us either.

Posted by bzimmy | Report as abusive

The government report is titled “Excessive speculation in the Wheat market”, which given the growth of long-only funds seems fair to me. It presents findings based on careful research and analysis of the appropriate model to use in evaluating commodities markets (and specifically addresses Nick’s fundamental value “by definition” approach).

You appear to discount the value of qualitative analysis. The Senate, however, is full of lawyers and is unlikely to agree with you in that judgment call.


I’m not sure I follow its certainty either. It says:

In a properly functioning futures market, futures and cash prices converge as futures contracts near expiration. Otherwise, if one price were higher, a trader could buy he commodity in the lesser-priced market and immediately sell it in the higher-priced
market for a quick profit. Those types of transactions would soon equalize the two prices. But on many occasions during the last few years in the Chicago wheat market, the two prices have not converged.

One key reason is that the large price disparity between the cash and futures price makes it much more profitable for grain merchants to buy grain in the cash market, hold onto it, and then sell it later—at the price of the higher-priced futures contracts—than engage in the type of transactions described above between the cash and futures market that would make the two prices converge. In addition, the large price disparity means
hat merchants who already have grain in storage and have hedged that grain by selling futures contracts could suffer a loss if they decided to actually sell their grain in the cash market, because they also would have to buy back the futures contract at a higher price
han they could get for selling their grain in the cash market.

but why doesn’t “grain merchants to buy grain in the cash market, hold onto it, and then sell it later—at the price of the higher-priced futures contracts” increase the spot price relative to the futures price? Indeed why are arbitragers buying spot wheat and selling it to the silly fools who want to invest in the higher price futures? The USO oil ETF has suffered from contango, but that hurts investors (ie those carrying oil), it’s good for the producers.

Posted by Matthew | Report as abusive

Sorry, penultimate sentence should read ‘Indeed why AREN’T arbitrageurs buying spot wheat…’

Posted by Matthew | Report as abusive

Agreed with Matthew–Csissoko’s (and the report’s) argument that higher futures prices could lead to higher expected future spot prices should mean that current spot prices go *up* if anything, since wheat is a storable commodity. The report is suggesting that there is a positive basis because holders of physical wheat are hoarding it, but if that is indeed the case, then the quoted spot price must not really be the market price because suppliers are unwilling to sell at that price.

On the discussions about econometric analysis, of course I don’t think that a regression alone can provide a conclusive answer to the question. A thorough investigation would use both qualitative analysis (such as interviews with market participants) and quantitative analysis (such as econometric modeling). While a model being consistent with the data does not prove that the model is correct, a model being inconsistent with the data is generally taken as strong evidence that the model is incorrect. This is exactly why statistical analysis uses the terminology “reject the null hypothesis” or “fail to reject the null hypothesis” rather than “accept the null hypothesis.” And one of the problems with this report is that it doesn’t even attempt to test its hypothesis against the data in a statistical framework.

I’d also like to expand on the point that the basis may be due to the oligopoly on physical delivery granted by the exchanges. Let’s see who holds the four delivery licenses, according to the report: ADM, Cargill, Nidera, The Andersons–large agribusiness companies that generally buy grain from farmers. If they are hedging in the futures market, they would be buyers of futures contracts. If futures contracts are closing above the spot price, that means Cargill et al are making money, so what incentive do they have to arbitrage the basis away through physical delivery? I’m not saying that my speculation is conclusive proof of this argument, just that the issue is worth thinking about. A positive basis at contract expiry is a risk-free arbitrage, so the fact that no one is executing it suggests that there are some barriers to doing so. And again, if what Congress really cared about was making the spot and futures prices converge at expiry, all they would have to do would be to require the contract to be cash-settled using the spot price as the reference price.

Posted by Nick | Report as abusive

I think the important paragraph is this:

“higher futures prices made it more profitable for grain elevator operators to purchase grain in the cash market, place it into storage, and then hedge those grain purchases with the sale of relatively high-priced futures contracts than to engage in arbitrage transactions (buying wheat in the cash market, selling futures contracts, and then delivering the wheat) at contract expiration”

But I don’t get it. The actions are the same initially – the elevator operators are buying (cheaper) grain in the spot market and selling forward (more expensive) grain in the futures. Then, however, the report says that they are holding onto the grain (so presumably settling the sold futures in cash) rather than delivering into them. But – and surely this is the nub of the entire argument – why are they doing this? It doesn’t seem profitable.

Posted by Matthew | Report as abusive

Matthew: Maybe the explanation is a combination of Nick’s and the report’s — there was hoarding and futures selling, but the average farmer was prevented from profiting because when storage reached its limits, the delivery oligopoly allowed the basis spread to go astronomical. (I know nothing about the market so this is pure speculation.)

Nick: “one of the problems with this report is that it doesn’t even attempt to test its hypothesis against the data in a statistical framework” This approach works in many cases, but some interesting models in the social sciences depend on unmeasurable variables like expectations. While using prices calculated from derivative contracts can proxy for expectations, they also introduce “technical factors” (i.e. market aberrations) with hard to assess biases.

Furthermore, it is important to understand that data analysis is extremely limited in what it can tell a social scientist, because many of the interesting questions involve counterfactuals — and the data to test them does not exist — unless you are willing to make strong assumptions about the event you’re interested in not being time and place specific (and therefore being sufficiently comparable to use data from “similar” events to analyze the event you’re interested in).

Do you really want to take the position that we should discard out of hand all models that rely on unmeasurable inputs? Personally I think that would be foolish in the extreme.


Csissoko–if one claims that the event of interest was so “time and place specific” that one rejects the possibility of using historical data to analyze factors at work, logical consistency would demand that one should also not use the analysis of the event in question as an input to policy decisions about the future, which is a different time and place than the event in question.

Posted by Nick | Report as abusive

Nick, My view is that while the human brain can incorporate very complex and subtle factors that make historical analysis useful when used with great caution, the idea that you can turn this extremely subtle understanding of events into a series of data points strikes me as utopian.


To be more precise a historian can compensate for underlying events such as changes in the legal environment, in norms in conduct of markets, in the number and quality of market participants, etc. In short the world is full of important, but unmeasurable variables.


The existence of the delivery oligopoly is necessary and nearly sufficient. That should be considered the primary cause of the failure here, even if it needed a trigger before the spreads got particularly wide in the last couple years. Making the futures cash-settled requires a way to determine the authoritative final settlement price; if it’s not too messy (in terms of performance risk) to dramatically expand the number of permitted deliverers, that would seem to me to be the sensible solution.


Post Your Comment

We welcome comments that advance the story through relevant opinion, anecdotes, links and data. If you see a comment that you believe is irrelevant or inappropriate, you can flag it to our editors by using the report abuse links. Views expressed in the comments do not represent those of Reuters. For more information on our comment policy, see http://blogs.reuters.com/fulldisclosure/2010/09/27/toward-a-more-thoughtful-conversation-on-stories/