"US vehicle miles traveled was affected"

Sure. That was partly an income effect (income going to oil sellers, some of them oil exporters), and partly conservation (reduction in low priority driving).

"Housing values started going down as well, especially in the most distant suburbs. "

Sure. That was also partly an income effect, as the poorest suburbs are the farthest out, and partly a result of the slowing housing bubble, which affected the newest construction, which was also the furthest out. Certainly it also was affected by rising commuting costs, but I don't see evidence that was the most important thing.

"I suppose if you follow some definitions, none of this affected world growth."

Of course it affected US growth. But, keep in mind that the income transfer from oil importers to exporters also shifted economic growth: the US & OECD slowed down, and Russia, FSU, & the ME grew more quickly.