National Hurricane Center and the Likelihood of Hurricanes
We do not, however, expect anything close to the U.S. landfalling hurricane activity of 2004.Well, they got that wrong...
This begs the question of how likely were the 2004 and 2005 hurricane seasons given the NHC forecasts? I'm going to argue that they were quite unlikely, suggesting the NHC, at least in recent years, is systematically underestimating the seriousness of the problem.
This is a simple high-school probability argument based on the following data. In December 2003 the NHC predicted a 68% chance of a major hurricane hit on the US (which I'll call a MLH - major landfalling hurricane), while in December 2004, they estimated a 69% chance of a major storm hitting the US coast.
In fact, in 2004 there were three major hits on the US (Charlie, Ivan, and Jeanne), and in 2005 there were four (Dennis, Katrina, Rita, and Wilma).
If we make one additional assumption - that the probability of any subsequent major landfalling storms is independent of the first and each other, then we can compute the probability of the observed outcomes given the forecast.
Specifically, the probability P that they give (the 69% or 68%) is the sum of the probability of getting one MLH, two MLHs, three, etc. We need to know p, the probability of getting exactly one, rather than P, the probability of getting one or more. P = p + p2 + p3 + .... If you remember algebra, and you imagine factoring a p out of the RHS, you should be able to see that P = p(1+P), so p = P/(1+P). To a good enough approximation for this purpose, 68% = 69% = 2/3. Therefore, p = 2/5 in both 2004 and 2005.
So what are the chances of getting 3 or more MLHs in a year? Well, p3+p4+.... That turns out to be about 11%. What are the chances of getting 4 or more MLHs in a year? Well, about 4%. If you want 3 or more in one year and four or more in the other, the chances of that are 4%*11% *2 (the factor of two comes because we would have viewed a reversal of the two years as equally significant). That suggests the overall chances of the 2004 and 2005 seasons, based on the forecasts, are 0.9%. If we had been willing to accept 2 storms one year and 5 another as equally significant (I wouldn't), that would raise it to 1.8%
The usual level of statistical significance is 5%, and the usual level for evidence to be "very significant" is 1%. Thus, the last two hurricane seasons constitute statistically very significant evidence that the forecasts understated the probability of major landfalling hurricanes in the US.