Wow, Dave, that's a lot of information there. Nice job pulling it all together. I think you put your finger on the situation best when you wrote:

Even without considering (in my view, probable oil shocks), our ability to predict future prices in the next year or two (on the front end) and price further out (on the back end) even considering market fundamentals and a structural adjustment in oil prices is impossible given the large number of independent variables that must be considered.
I agree with this, that predicting these prices is fundamentally impossible given the uncertainties.

I should mention, too, that I am not an economist. My background is actually not all that different from Stuart's: I work in cryptography and software design, and I have a bachelor's degree in engineering from Caltech. However I have a great respect for the economist's way of thinking, at least as I define it.

Not everything that is conventionally thought of as 'economics' falls into that category, though. I think a lot of the material you quoted is pretty much BS. All these people claiming to know the future, to know what the stable price of oil is! It's junk economics, IMO. I've posted before on studies showing that so-called expert predictions are generally pretty worthless. Highly-paid economic advisors can't beat chimpanzees throwing darts. Most of these guys are operating an enormous scam, a confidence game. I think a lot of the people you are quoting, the ones who claim to know the future, fall into that group.

Where I look for reliable information is the markets themselves. And what the markets are telling us, as in those tables you quote from JDH (a good economist, btw), is that the future is highly uncertain! Look just three years out and ask for 95% reliability, and all the market can tell you is that the price will be between $20 and $180! I worked out the figures for a narrower spread, and if we want just 68% reliability, the range is still $34-104. There's a full 1/3 chance we will be out of that range by the end of 08. So the markets are actually very consistent with your statement I quoted above. They are telling us that the future is highly uncertain, so-called experts notwithstanding.

Where I look for reliable information is the markets themselves.

You are half-way to enlightenment ;-).

Prices just are.

Or they are some mix of signal and noise, and no one really knows how much is signal and how much is noise at any point in time.

Although no one knows how much of the price estimates are signal and how much are noise, the market allows you to bet on the question! That's what options are for, in part, to provide information on the degree of uncertainty in market estimates of future prices. It is by looking at option prices that JDH was able to create that table showing 95% confidence range for market prices.

Options let people bet on how much is signal and how much is noise. If you disagree with the market consensus on the question - if you think the markets are over- or under-estimating the degree of certainty in a price estimate - you can take a position in the market and make a profit if you are right.

I don't want to get too far ahead in my answer, because I'm not sure what your focus is.  People bet on things that are purely noise (roulette) ...
BTW, we should probably note the origin of that 95% confidence level:

One implication of the random walk model is that, if the quarterly change in the log of the real oil price has variance s2, then the variance of the change over two quarters, being the sum of two uncorrelated variables, has variance 2s2, and the variance of the change over a year would be 4s2. The quarterly value of s is estimated from the 1970-2005 experience to be around 0.16, meaning a forecast of the real oil price a year from now would have a standard deviation of 40.5 x 0.16 = 0.32-- it's not that uncommon for the oil price to change by 32% on a year-to-year basis. If we use plus or minus two standard deviations to form a 95% confidence interval and convert from logs back to levels, we arrive at the confidence ranges implied by the table on the right below.

if the 1970-2005 experience is representative of the near term future, then ...

We can put ourselves back at some points in history where such projections would be less safe than others.  If we'd calculated in 1970 for instance, we'd get a very different idea of oil price variance than we would later, in 1980.

Q: Does anything related to "peak oil" imply a change in variance?

JDH goes on to say in that article:
Another way to judge how uncertain experts are about where oil is headed is to look at the volatility that is implicit in crude oil options prices. Using the Black-Scholes formula, one calculates an implied volatility from current values of options on NYMEX that corresponds to an annual standard deviation of 32% -- the identical number as from the historical volatility above.
This data comes from option prices (that's what the Black-Scholes formula is about) and allows us to directly read out the market's opinion of the range of likely future prices.

So at this point, it appears that the answer is no, considerations of "peak oil" are not changing the market's opinion of future volatility.

This is the point I was making above, that the market not only gives us price estimates, it also gives us error bars on those prices. You can bet on the prices or you can bet on the size of the error. Either way, if you have a strong suspicion that the market is getting it wrong, you can profit. If you think that variance should increase in the next few years due to greater volatility caused by Peak Oil, you can take positions in out-of-the-money options, which conventional market thinking will set as underpriced.

I made the point earlier that actually, most people here are in luck. They live in a world in which money is just lying around for the taking. Since most contributors here strongly disagree with market prices, either in absolute terms or in terms of the chances of hitting certain extreme prices, they see a market that is full of low-risk, high-profit opportunities. It's as though you could buy a lottery ticket where you had a 95% chance of winning, instead of 0.00001% or whatever. That's how the world looks for a Peak Oil true believer.

People in the markets are foolishly giving their money away! And all it takes to get rich for the Peak Oiler is the courage of his convictions. Oh, and he has to be right, of course. But most people don't have much doubt on that score. At least, not until they have to actually put money down, and they suddenly realize that there must be some reason the other guy is willing to bet against them...

As long as people note that the cardinal rule of investing is that "past performance is no guarantee of future results," then sure, you can calculate a variance one way or another.

You can "check" future performance with a Black-Scholes calculation based current prices, but such a check still carries an assumption.

I'm afraid I also don't follow how a 95% chance of a price "between $20 and $180"(*) translates into a 95% chance of winning.

* - assuming current market characteristics hold, and that no "game changers" appear

The 95% winning lottery ticket is with regard to people who think that market prices are wrong. Surely you have seen, as I have, many postings here in recent days commenting about how unreasonable and illogical market prices seem to be.

As an example, look at Fletcher's posting below:

http://www.theoildrum.com/story/2006/3/2/234845/7384#23

and my response. He predicted that oil would not fall below $55 again. I pointed out that the markets saw a 30% chance of that happening before 2009. If he is 95% sure he is right, he can get free money from the market, just like a 95% winning lottery ticket. He can cash in for six thousand dollars a contract with, in his view, almost no risk.

Another example was a few days ago:

http://www.theoildrum.com/comments/2006/3/1/3402/63420/367#367

Don predicted 5 to 3 odds that oil would hit $200 by 2010. The markets say the odds are more like 1 in 20! I showed him a contract that would have probably a 95% chance of being profitable (in his judgement) and that would have a better than even chance of making him 20 times his investment! How's that for a 95% successful lottery ticket?

This leads to a point I made yesterday. It's fine to come up with your own opinions on the odds of various events. But once you are informed of the market estimate, if it differs from your own view, you have a rather uncomfortable choice. You can change your estimate to match that of the market. That's what I do.

Or, if you insist on holding to your own ideas and believe that the market is wrong, you have to accept that there is free money available that you are passing up. There are positions you can take in the market that will be profitable with very little risk. The greater your disagreement with the market, the easier it is to find such positions and the greater the profitability, as in the examples above.

Since it is of questionable rationality to pass up free money, it follows that it is questionable whether it is rationally possible to disagree with the markets. It could be that when you are informed of a market consensus, you are more or less forced to agree with it! Sounds bizarre, but I think there may be some results in the economic literature along these lines. It is an area I want to look at in more detail.

Maybe I'm generous, but I think that people posting prices here are just taking a semi-serious pass at it.

Or you may remember the way I divided it, anybody who names "a price" is either:

  • assuming no wildcards (not a good bet with real money)
  • think they know the wildcards (fools)
  • or are just taking a swipe at it (without real conviction)

I don't really think you should push the "true believer" thing too hard, because it rapidly becomes a straw man.  A true believer has to be someone who:

  • believes "peak oil" is true
  • believes that "peak oil" will be the strongest market factor in the next 10 years
  • believes that no wildcards short or long term will upset this applecart

So really, a "true believer" has to understand oil, but not markets.
Just to name one wild card combination, what if peak oil is (broadly speaking) true, but China simply decides (through command and control) to reduce its oil use in absolute terms?
Again, you're assuming that price development happens in a predictable and direct manner. Instead, there are huge swings in both directions. Economists assume that there is a fair price (equilibrium). The market never thinks that. Betting on a certain price development will always fail if it is a statement about equilibrium. If it is an attempt at catching swings, it might work.

To say that the price of oil will never fall bellow $50 should be made very carfully by a trader. As an economist who adheres to Peak Oil, I would claim the same. As a trader, I would even be willing to bet on falling prices, although not necessarily at the present.

That's how the world looks for a Peak Oil true believer.

A True Believer is not necessarily a True Trader. Being 95% certain in the mid to long range on the markets doesn't help much. Entrance and Exit points on the market do. On the market: 2 + 2 = 5 (-1). Its more like dancing, not like empirical science.