Now to get to the juicy bits - the equations! There are two deterministic products you can derive from your data. I did similar work in an undergrad thesis for a complex system that seemed not directly controllable, yet when the empirical data was accumulated, graphed, time shifted to compensate for lag, out popped the relationships in;

1. Transfer function: as stated, this is a typical lagged feedback control loop. Using the standard controls systems engineering tools, you can determine the system response in the frequency domain. This will allow you to determine the system response due to set points and forcing functions. The break-even price is a definite set point that has a y=ax+b function over time y=$, a=constant in $/time, x=time, b=$.

2. Equation: P(price) = f(d,D,S,...) d=drilling rate, D=# drilling rigs, S=storage, etc. By performing a multiple regression analysis, you will derive a linear algebraic function with a degree of correlation (R). The higher the value of 0

From the viewpoint of predicting more or less volatility, what factors do you think are most important? It seems that spikes are increasing in intensity, but not in frequency. We do know that several things have changed during this decade that might increase the sensitivity to spiking:

1. Depletion happens faster
2. The drilling fleet is much larger
3. Electrical generation use is growing
4. Industrial use is falling (although I think this would reduce the spiking rate).

I branched comp sci instead of EE and missed taking a control systems class. I have regretted that several times. If you have a bit of time I would be glad to provide the data in spreadsheet form. Just shoot me an email at my profile address. Otherwise, can you recommend a good introductory article?

Thanks Jon, I was going to ask if we could get the data set. In our group of EE's and controls systems engineers, I am probably the least talented, but I see the bigger picture and I'll explain the knock on effects. That is, why is this information is important across multiple industries.

Natural Gas electrical generation (i.e. best Scotch to wash dishes) has become the baseline cost for many electric utilities. When forecasting electrical rates for long term planning - at least on the West coast - natural gas rates are projected to provide the reference pricing. The LTAP's (Long Term Acquistion Plan) I've seen lately use the EIA projections; and we all know the track record on either USGS or EIA prognostications!

I believe the real deterministic near-term and longer term trends lie in the empirical data sets you have presented. Therefore, I got the ball rolling this morning after reading your report to bring in our control systems and signal systems engineers to produce the equations I mentioned earlier. Producing a multiple regression equation should be the simpler of the two, while the transfer function and control block diagrams will take some trial and error. Furthermore, the inputs to the transfer function may require Convolution transforms to adapt the "signal" waveforms into coherent functions.

But, in the end I would expect we will have a set of equations that can be utilized to correlate cause and effect. I like the NG market because it is mostly homogeneous, mostly closed, and decoupled enough to allow independent variables to respond to non-correlated signals. On the other hand, I don't think this analysis practice would be applicable to the oil market for all the opposite reasons.

Will email in a day or two once we get a handle on the scope of our analysis and outputs. Again thanks, I've found your reports very informative, and this is probably because you are outside the industry and can look at it like it is a black block (Boy, that's leading right into the Entropic Principle - whew!).

Note that files can be attached to a posting.  That would be a good place for a spreadsheet.

Good idea. This is a link to Jon's spreadsheet.