Web -- the best insights I took away from the "Black Swan" were not so much about BS's but the misuse of data to which you refer. Especially the way "the story" is used to support the interpretation of the data. For those who haven't read Taleb's book his point has to do with "spin". He points out that such spin does more than reflect a view point. For the majority of folks it's adds a context to the data which in turns makes the interpretation more valid then it actually merits. Simply put we tend to remember and accept the story about the data more than the data itself. After reading this aspect in the BS I noticed how very common this approach is for everyone with a point to make...including myself. Let's face facts: the great percentage of chats on TOD don't deal with the "data" but our conversations about the data.

Yes, Taleb calls this the "narrative fallacy" : our need to fit a story or pattern to a series of connected or disconnected facts. The statistical application of this is data mining and how the results get interpreted. Taleb sends mixed signals on this. He lauds intuitive empiricism, but only if done correctly. He doesn't say how to do it correctly though (he is the expert on such matters). By the same token, he does not like statistical modeling, unless it borrows ideas from statistical physics. That recommendation on page 267 made my day, as that is all I try to use.

Web -- What is the recomendation? I only have it on a cd.

ROCK - here's a link to the page in question (or close to it). You could search your CD doc for the word "Ormerod" and get there.

That's the one. Taleb essentially recommends that we use real physics-based statistical models while "avoiding econometrics and Gaussian-style non-scalable statistics like the plague."

This is completely a fresh way of looking at the problem, and one that I had been developing the past few years. I knew kind of what was in the Black Swan book, but it wasn't until I finished reading it in the last couple of days that I completely understood Taleb's mind-set. Most everyone that has referenced the book have drawn different conclusions about its deeper meaning. This is partly because of the way the book is written -- very personal, blunt, and also allegorical. This book is so blunt in fact, that if it was written as a series of blog posts on TOD, they might actually get deleted. No one is immune to the criticism of NNT :)

Mucho thanks KLR. Not sure if what I picked up was Web's point but what caught my eye was his position about folks using models as predictors and not just a descriptive mechanism. I've debated others here in that regards. To me all models are correct (as long as the construction is proper). For me a model is just a formula with all the assumptions filled in. One can argue if the assumptions are realistic. Regardless the model does provide an "correct" answer if designed properly. That doesn't mean that correct answer is a prediction. Just that, for this set of assumptions, the model is correct. Where I see folks make huge mistakes is offering such models as predictions. For me the value of a model is its ability to show a system's sensitivity to the various components. A model that shows the great sensitivity of oil prices to global economic activity is one example. Another would be the relatively low sensitivity oil prices might have relative to new production from the Bakken Shale. IMHO where folks stub their toes is when they make predictions of the future. My argument might sound subtle to some but it explains why I don't care to debate the validity of predictive models. For the most part they are not close to being reliable even when I think the model is constructed properly. And then, as Web points out, to make matters worse folks will use the conversation of the model as spin to support their proposed accuracy of their prediction. And the spin suddenly offers more credibility then the "model" deserves.

Along the same lines, the spin thing is the main reasons polling data is what it is. Spinning the construction of the questions, their ordering, the type of responses recorded and on and on can steer the result very sigfnificantly toward the direction whoever is funding the poll most prefers. Modelling is all similar in that way, if you hang a bunch of fabric on a skinny 5'11" runway strutter you aren't going to get a very good prediction of what it will look like on the average 5'4" big mac butted couch sitter ?-)