As you implied, the problem is the number of wells that we have to drill. For example, Texas natural gas production, from gas wells, in 2006 was the highest we have seen since 1981, but it took more than twice as many gas wells in 2006 to produce the same amount of gas that we produced in 1981:

http://www.rrc.state.tx.us/divisions/og/statistics/production/ogisgpwc.html

I think that the fourth quarter of 2007 was the real start of the permanent energy boom/crisis. Whether it is a boom or a crisis depends on what side of the producer/consumer equation that one is located on. The industry can and will make money from exploiting smaller conventional oil and gas fields and from exploiting unonventional resources, but the question is whether we can increase our total net energy output, especially with personnel and equipment constraints.