There is a Harvard Professor whose name completely escapes me, who has revitalised urban economics.

Gleaser I think.  Edward Gleaser.

He has published pretty widely on this point.  Until 1970, US housing prices generally rose about with real incomes, housing didn't get more or less expensive over time (although mortgage financing became easier).  Taking the country as a whole.

Since then, there has been a noticeable split.  Basically on the coasts, housing prices have risen much, much faster than incomes-- especially Boston, SF-Bay Area, LA, NYC, Washington.

In the inlands, prices have continued to rise at about real incomes (the last 4-5 years have been crazy and one needs to discount those).  And so most population growth has been in Sunbelt cities like Atlanta and Phoenix that have mushroomed in size.

Gleaser's analysis attributes 70-80% of that increase in real prices to NIMBY/ zoning controls.  Basically in places like Phoenix AZ, they just keep building new subdivisions.  And in the 1920s in NYC, townhouses were replaced with 12-20 story apartment buildings.

But what has happened in the 1980s and 90s in NYC is those Upper West Side apartment blocks have not been replaced by 50-60 story buildings.  Whereas in a City like Hong Kong, with the same space constraints and explosive growth, they have.

Needless to say the same pattern is reflected in Queens, and then again in Nassau County, Long Island etc.  The battle is being fought over the mayor's new development in downtown Brooklyn, and indeed they have reduced the size of the projected development (by about 8%).

I can't help but wonder, though, what would happen if they were to take that land in Brooklyn, divide it up and sell it to many different people, so you get the same amount of housing, but not built in one huge ugly city-excluding glob by the developer. I think to some extent the objections are not to the amount of housing but to the fact that it's all built as a single huge project by a single developer, as a self contained unit with no regard for the city around it.
I don't know enough about the local issues, but I agree the 'superblock' scale of the thing is part of what is so offputting (but also increases the achievable density).

If I look at other places (the Barbican complex here in London, the original WTC site) then the 'super block' has not been a successful concept.