http://www.vision42.org/ a proposal for a 42nd street street car.

I suspect the labour unions issues with the New York subway would need to be fixed as well, to meaningfully increase the thruput without a big expansion of cost.

Nope. All they have to do is increase the acceleration of the trains. If a train has 50% faster average speed, then it completes its run in only two thirds the time. So the same train and train crew can complete 50% more runs in the same time, and thus carry 50% more passengers for the same cost.
The real problem has been that the MTA hasn't been willing to either adjust the signal system to allow faster speeds, or install a normal ATC system of the kind that, say, the LIRR has. Instead what they had was CBTC, which is all fancy and computer controlled and works by radio, and they're going to put it in service on the L train sometime in Spring 2005. Which came and went with no CBTC. And the MTA just doesn't seem to realize that running trains faster is possible, or that it is useful and important both for increased productivity and making the system a more attractive option. Oh, and if you want to see trains that go fast, just take a trip to Philadelphia. Ride the Broad Street Subway, especially the express, and the Market-Frankford El, and if you have time, PATCO and the Norristown line are also worthwhile. The subway and PATCO has a speed limit of 70, and the El gets up to 40mph between 11th and 13th street station. For comparison, the typical NYC subway express train on level track can get up to 35-40 mph depending on the train and how long it has to accelerate, and local trains can hit 30 right before starting to brake coming into a station.
Our rails go out of alignment in hot weather, so there are speed restrictions across both the rail and Tube networks-- down to 15mph I think.

The Jubilee Line extension was supposed to be on a 'moving block' signalling system, rather than the 'fixed block' that the Victorians invented.  The software never worked properly.

Based on the British experience, the safety cost of one bad accident, in terms of what the rest of the network is forced to spend, is so large that I can understand why operators are cautious about increasing speeds.

The tube is in really terrible condition, which is why it has all those speed restrictions and so on. Reliability on the Tube is terrible, worse than the NYC subway even, which is surprising. There are ways to deal with heat expansion of rails, for example expansion joints which are used successfully on everything from Japan's Shinkansen to Philadelphia's Market-Frankford El. And it seems that the people in charge don't really want to invest serious money in fixing the basic infrastructure. Instead they tend to invest in whiz-bang technologies like moving block (the same thing that was installed and failed to work on the L train). Incidentally, the guy in charge of that particular project in NYC now works as a consultant for the company that sells that technology. Oh, and the technology had problems not with the software but the fact that it depended on radio, and specifically timing of radio signals, in tunnels. It's just physically nearly impossible to do that reliably.
Oh, and please don't bring out that "technology that the Victorians invented", because after all, most transportation uses "wheels" which were invented by cavemen.
Here in Vancouver, a fully automatic, grade-separated light rail system called Skytrain has been in operation since 1985. It uses a moving block system that uses wires instead of radio signals. During rush hour, trains run with a frequency of 1 train every 104 seconds on a portion of track where two lines are combined. The system carries over 200,000 passengers on a typical week day. The top speed is 80 km/h (50 mph) and it's very reliable. Average speed including stops is about 45 km/h (28 mph).

Oh yes, it did cost a lot of money to build, but the operating costs are low.

The "Skytrain" system is basically the same technology and very similar construction to the JFK Airtrain, though the Skytrain I think has just regular platforms instead of enclosed ones with doors. Incidentally, the East Side IRT used to run 35 trains per hour, with plain old three-aspect fixed block signals with mechanical trans stops and no automation, which is roughly the same traffic density as the Vancouver system, and in fact manual operation is necessary for any higher traffic densities, because automation still is not very good at stopping trains precisely, quickly, and under a variety of conditions. Oh, and while people may call the Skytrain "light rail", it really is a proper rapid transit system, belonging to some intermediate class between light rail (interurbans) and full heavy rail metros, and it has more in common technically with the latter than the former.
This initial Skytrain line was built in part because Vancouver was hosting the 1986 tranportation expo. There was enormous opposition to the initial line and several later extensions. Many believed that very few would dare riding a system that didn't have drivers. This concern was proven unfounded soon after it opened. There was also a lot of talk about crime, destruction of neighborhoods, etc. After many battles, the system was extended 3 times, the latest extension was completed in 2001.

The platforms are open, so you can easily fall onto the tracks. This happens on occasion and has led to several deaths. There are track intrusion system at station that are supposed to stop an approaching train, but I'm sure how well they work. The older stations use touch pads, while the newer ones use laser beams.

There has never been any collisions between trains, except some minor fender benders in the yard where the trains sometime run on manual.

The weird thing about riding an automated rail car is the consistent ride. The train stops a precicely the same spot at a particular station, and speeds up and slows down at exactly the same spot on the track everytime you ride it.

The system is somewhat like a sideways elevator. The train won't move if someone is jammed in a door. Passengers often take gross advantage this safety system by forcing the doors open to stop a train that is just about move. This is why the design frequency of 1 train every 90 seconds has not been sustained in practice.

I think that it's very difficult to design a system that is both frequent and fast. You can increase the frequency by simply going slower, but this doesn't sound a good thing.

Despite its many detractors, I would judge the Vancouver Skytrain system a resounding success. It has created several attractive, high density neighborhoods that would otherwise not exist.

Another automated rail system is currently being build that will connect downtown to the airport and a southern suburb. The control system will use moving blocks, but the vehicles won't be powered by linear induction motors like Skytrain. It's supposed to be ready by the time we host the 2010 winter olympics.
 

The most frequent subway service I know in the world, in Moscow, runs 39 trains per hour per direction, for a headway of 92 seconds. The top speed there is also 80 km/h, and the average line speed is about 40 km/h, so that's comparable to Vancouver. The signal system is a fixed block system with automatic speed control via coded track circuits, with wayside block signals with train stops as a backup. Trains are usually 7 or 8 cars, with some lines running 6 car trains. The train cars themselves are about as narrow as the IRT cars, but slightly longer (62 feet). The Moscow subway system of 10 lines carries around 9-10 million riders daily, with the busiest line getting somewhere between 1.5 and 2 million of that. The single busiest station (which happens to be the only cross platform subway-railroad transfer in Russia) has a daily ridership of about 250,000.
I agree with the preference for 'fancy pants' solutions over solid basic rail engineering.

The problem is the middle management layers have been decimated by redundancy, early retirement and 'reorganisation'.  The people at the top don't know the technology, don't know what works.  And they are selling to politicians who want fancy solutions.

Also there is a fundamental disrespect of technological people and technical solutions: nerds, boffins, dweebs, trainspotters... pick your epithet.  This is a very British thing, but I've seen it in Canada, too.

I've seen this in the power generation sector.  Virtually any heavy industry, the 80s and 90s devastated the human capital-- no one young and bright went into them.

Countries like France still have that technical capability, one of the reasons the French nuclear system performed so much better than the British.