It depends on engine speed as well. Internal combustion engines reach their peak efficiency at a particular engine speed which is usually not the slowest smooth speed. I seem to remember that for typical petrol (gasoline) engines it's about 2500 rpm.

There's really no hard fast rule about the ideal speed. It all depends on the car.

My car (2002 Camaro V6) has a sweet spot in 5th gear at about 2400 rpm. On the highway that equates to about 70-80 mph, depending on the terrain. At lower speed in 5th gear, I don't have enough torque to pass safely and I have to row alot between 4th and 5th. Any higher and I'm inviting a ticket.

My car is EPA rated at 19 city/24 highway and I average about 21 in the city and 32 on the highway.

It helps to distinguish between potential most economical speed and realized most economical speed.  You're right, in that the best economy you can get would be the lowest speed that you can maintain in your highest gear without lugging the engine.  That's the speed that minimizes wind drag, engine drag, road resistance, etc.  The record efficiency tests, where you hear about some Volvo or other car getting some stupendous mileage on a test track, involve someone getting the car to 30-35 or so, and leaving it there on cruise control for hours as they drive around a track.

The problem with that is that most people end up constantly speeding up and slowing down, in which case you end up lugging the engine pretty badly at such a low speed.  So people tend to get better mileage at a slightly higher speed.  Also, some automatic transmissions work differently at different speeds, so the torque converter may not lock until you reach a certain speed.  The locking of the clutch improves fuel economy at that point and above (see torque converter at howstuffworks.)  Automatic and manual transmissions are different in terms of most economical speed.  

Our wagon with a manual transmission gets its best fuel economy at about 30mph, but you have to pay constant attention to the amount of gas you're giving it and the terrain/conditions ahead.

Back when there were competitive economy runs I entered them with some success.  My rule of thumb was to drive as much as possible in top gear at the engine's specified torque peak (given in the owners manual), usually something in the vicinity of 2500 rpm. This no longer works.  My current car has a six-speed manual gearbox and seems to get better mileage in fifth gear because the torque peak in sixth is too fast for good economy.