Weeeelll. I think part of the issue is this: there's a number of potentially pressing pieces of the problem:

1. Oil supply might decline faster than the economy can tolerate due to high decline rates and we might end up facing depression style economic conditions in the not too distant future.
2. Large fractions of the oil supply might go offline due to geopolitical problems
3. We might fail to ever develop substitutes for oil that will allow us to have anything like our current civilization, particularly since we'll leave our grandkids with a world with fewer other undegraded resources left.
4. The climate might go to hell (eg undergo an abrupt flip to some completely different state, as it has in the past).
5. Some developing countries might start to fail really badly causing overwhelming human suffering and refugee problems

I think reasonable people can differ about which of these are the greatest risk. I suspect Matt Simmons is mainly worried about 1) and 2), which tend to make drilling ANWR look like a good idea. I suspect you are worried about 4), which argues for leaving everything possible in the ground. I think Roscoe Bartlett is driven a lot by 3), which tends to also argue for leaving ANWR to our grandkids (especially since he has so many of them!). Personally, I can't decide which to worry about the most, hence the fence-sitting behavior that is galling you :-)