A few basics definitions are required for the newbies in commodity trading (including myself):

• Contracts: covers 1,000 U.S. barrels for crude oil. For crude oil, each contract expires on the third business day prior to the 25th calendar day of the month preceding the delivery month. If the 25th calendar day of the month is a non-business day, trading ceases on the third business day prior to the business day preceding the 25th calendar day. After a contract expires, the front contract for the remainder of that calendar month is the second following month.
• CFTC: The Commodity Futures Trading Commission (CFTC) is a federal agency that is collecting data on the composition of open interest for all futures contracts. The CFTC is charged with regulating futures and options trading to ensure that the markets are free from manipulation.
• COT: The Commitments of Traders (COT) reports provide a breakdown of each Tuesday’s open interest for market reports in which 20 or more traders hold positions equal to or above the reporting levels established by the CFTC.
• Open Interest: Open interest is the total of all futures and/or option contracts entered into and not yet offset by a transaction, by delivery, by exercise, etc. The aggregate of all long open interest is equal to the aggregate of all short open interest.
• Reporting Commercial traders: are associated with an underlying cash-related business and they are commonly considered to be hedgers.
• Reporting Non-Commercial traders:  are not involved in an underlying cash business; thus, they are referred to as speculators (commodity funds). Boone Pickens, well known in the Peak Oil community, is also the head of one of the most successful hedge fund ever (BP Capital Management).
• Non reporting traders: small speculators.
• Spreading: For the futures-only report, spreading measures the extent to which each non-commercial trader holds equal long and short futures positions.
• Short position: When trading futures contracts, being 'short' means having the legal obligation to deliver something at the expiration of the contract, although the holder of the short position may alternately buy back the contract prior to expiration instead of making delivery. Short futures transactions are often used by producers of a commodity to fix the future price of goods they have not yet produced. Shorting a futures contract is sometimes also used by those holding the underlying asset (i.e. those with a long position) as a temporary hedge against price declines (src: Wikipedia).

The principle of trading futures is briefly explained here:

Trading crude oil futures is done the same way as any other investment - by buying low and selling high. One difference with futures, however, is that it's just as common to sell "short" - to sell first, in other words - and then buy back later as it is to buy first, or "go long." With futures trading, if you think prices are going up, you simply establish a "long" (buy) position. If you think prices are going down, you initiate a "short" (sell) position. Once you've established your futures position, you have a couple of alternatives:

• Offset your position by taking an equal but opposite position. You can exit from any futures position before the contract expires by taking an equal but opposite futures position (selling if you have bought; or buying if you have sold). Most futures are offset in this way. You don't have to wait until the expiration date to complete your trade - in fact, few investors do.
• "Roll" the position over from one contract expiration into the next. If you hold a long position in an expiration month, you can simultaneously sell that expiration month and buy the next expiration month (known as a "calendar spread") for an agree-upon price differential. The opposite is also true - you can roll a short position from the expiration month to the next available trading month just as easily. By transferring or "rolling" a position forward this way you are able to hold it for a longer period of time. For example, if you are holding a June Gold futures contract, you can sell the June futures before expiration and buy a December Gold futures contract, thereby expanding the timeframe of the trade.

The main findings of the Senate report can be summarized as the following:

1. Rise in Speculation. Over the past few years speculators have expended tens of billions of dollars in U.S. energy commodity markets.
2. Speculation Has Increased Prices. Speculation has contributed to rising U.S. energy prices, but gaps in available market data currently impede analysis of the specific amount of speculation, the commodity trades involved, the markets affected, and the extent of price impacts.
3. Price-Inventory Relationship Altered. With respect to crude oil, the influx of speculative dollars appears to have altered the historical relationship between price and inventory, leading the current oil market to be characterized by both large inventories and high prices.
4. Large Trader Reports Essential. CFTC access to daily reports of large trades of energy commodities is essential to its ability to detect and deter price manipulation. The CFTC’s ability to detect and deter energy price manipulation is suffering from critical information gaps, because traders on OTC electronic exchanges and the London ICE Futures are currently exempt from CFTC reporting requirements. Large trader reporting is also essential to analyze the effect of speculation on energy prices.
5. ICE Impact on Energy Prices. ICE’s filings with the Securities and Exchange Commission and other evidence indicate that its over-the-counter electronic exchange performs a price discovery function – and thereby affects U.S. energy prices – in the cash market for the energy commodities traded on that exchange.

These two charts below are seen as "evidence" of speculative activities in crude oil and natural gas markets because of the correlation between Non-Commercial spread positions and prices



However, correlation is not causation and Professor Graig Pirrong (Professor of Finance and Energy Markets, University of Houston) has offered a nice rebuttal to this report on his blog. He blasted the report as being "a farrago of facts, factoids, and falsehoods stitched together to arrive at a conclusion", in his view:

Relationship Between Traders' Positions and Prices

This recent study by Sanders et al.:

Dwight R. Sanders, Keith Boris, Mark Manfredo, Hedgers, funds, and small speculators in the energy futures markets: an analysis of the CFTC’s Commitments of Traders reports,
 Energy Economics (26), 2004, pp. 425– 445.

looks at the relationships between market returns and trader positions (buying short or long contracts) and came to the following observations:

1. Traders classification is somewhat imprecise and subject to errors: there is no information about the motives of no reporting traders, commercial positions likely reflect a diverse set of motives. However, non commercial traders have no incentives to classify themselves as commercials.
2. Between 1992 and 1999, non commercial positions are a small fraction of the TOI (~10%) but they are very volatile (i.e. they change from long to short on a weekly basis) whereas commercial are a large fraction (~70%) and are buying short positions.
3. When prices rise, reporting non commercials are net buyers of long positions (i.e. they tend to follow trends and act as a group) whereas commercial traders are sellers of long positions.
4. When prices fall, reporting non commercials are net sellers whereas commercial traders are long positions buyers.
5. Commercial traders's positions do not lead price (funds do not increase long positions prior to rising prices).

Unfortunately, the study used only data for 1992 to 1999 and we don't know if the above observations are still valid. The market total open interest (noted TOI) can be separated in different quantities given in the COT report:
where NCL, NCS and NCSP are non commercial short, long and spreading positions respectively, CL and CS are commercial long and short positions, NRL and NRS are non reporting long and short positions. From these parameters, we can derive several position measurements:

Reporting commercials percent of TOI = (CL + CS) / (2TOI)
Reporting non commercials percent of TOI = (NCL + NCS + 2(NCSP)) / (2TOI)
Commercial NPL= (CL - CS) / (CL + CS)
Non commercial NPL= (NCL - NCS) / (NCL + NCS + 2(NCSP))

The commercial PNL (Percent Net Long position) is also called "hedging pressure" and the non commercial PNL is the "speculative pressure".

The figures below are showing the importance of each trader category as defined by the CFTC, we can see that the Non Commercial and the Non Reporting groups are a small portion of the total open interest and that their activities have significantly changed since 2002 (see Table I below). Positions held by non commercials have increased with prices and are now around 40%, we can see clearly that changes in positions or non commercials and commercials are antagonists.


Fig. 3 Percent of total open interest per trader category. Click To Enlarge.
Another characteristic is that non-commercial traders were switching from long to short all the time compared to the commercial group before 2003 but have consistently stayed long since 2003 which can explain the contango situation in the future market.


Fig. 4 Percent of total open interest per trader category. Non commercial PNL is also called "speculative pressure". Click To Enlarge.
Table I. Percent of total open interest and Percent net long held by CFTC reporting categories.

		1996-2002	2003-2007
Reporting commercials percent of TOI (%)	Commercials	69.8a 4.0b (61.3,  78.3)c	61.5 3.5 (53.4, 68.9)
	Non Commercials	18.3 4.19 (9.63, 29.2)	32.8 4.6 (24.1, 42.1)
PNL (%)	Commercials	-1.71 4.41 (-13.0, 14.2)	-3.63 3.19 (-33.4, 35.5)
	Non Commercials	12.8 5.55 (-33.4, 35.5)	8.0 6.22 (-10.34, 25.0)

^aaverage over the period.
^bstandard deviation over the period.
^cThe minimum and maximum sample values are presented in parentheses (minimum, maximum).

Futures Trading and Inventories

There is also a strong relation between the speculation on future contracts and inventories levels. When future prices are higher than the front month contract (a situation called Contango), it becomes profitable to keep inventories high because you are guaranteed to make a profit. Conversely, when futures are lower than short contracts (a situation called backwardation), inventory levels will tend to decrease. The strong run up in prices since the beginning of this year has push front prices faster than future prices and we are now in a backwardation position. In addition, stock levels have started to fall.


Fig. 5 Charts from the Wall Street Journal (Where Has the Oil Gone? October 6, 2007).
Prof. Hamilton and Pirrong have also recently commented on this situation (here and here).

All of this illustrates how complex the energy markets are, the mechanisms are very often oversimplified by medias and appearance can be deceiving. In particular, we have to be cautious in our interpretation of the stock levels. A lower inventory situation does not necessarily reflect a real supply/demand inadequacy. 

Related stories on TOD:

Gail the Actuary, What Can the Commodity Market Tell Us About Peak Oil?
Dave Cohen, Scarcity and Oil Prices Again
Dave Cohen, Predicting Future Oil Prices
Nate Hagens, A Closer Look at Oil Futures
Stuart Staniford, Is Oil in a Price Bubble?


And when you're done with this, don't forget to check out Robert's This Week in Petroleum