Lonesome Trail, to some degree I mean both, but mostly the former. Here's a BSFC chart for my car:
I'm not sure how familiar you are with these sorts of charts, but it's an RPM vs torque chart, with the amount of fuel needed to produce a given amount of power printed on contour lines over the load and rev range. Lower fuel consumed to make the same amount of power is better.
I'm going to draw some lines on this chart, and then explain them:
One important thing to note about this chart is that it is not a power (kwh or HP) chart - as you move from left to right,
horizontally, you have a roughly linear increase in horsepower, because power = torque * RPM.
Another important thing to note is that the contours are elongated horizontally. Increasing or reducing load has more impact on efficiency than RPM does, meaning it's nearly always better to upshift.
Hypothetically, let's say you shift up at about 2500rpm, as I do, and the next gear brings you down to about 1750rpm. You accelerate at around 75% load, which you can determine with instrumentation, such as a vacuum gauge or scangauge. By doing this, you'll be moving back and forth alone the green line, staying in the engine's peak efficiency range. How long it takes you to accelerate is irrelevant, but ideally, you want to spend the greatest amount of time in the highest efficiency range.
The red line I've drawn represents (very roughly) a fixed amount of horsepower. If you need 20HP to cruise on the highway, you could do it at 6000RPM with 10% load, resulting in an incredibly poor BSFC of over 800, or you could do it at ~2000RPM with 80% load whit a BSFC closer to 200, which is 4x more efficient.
The only time taller gearing is not better is when you can't use it because you simply can't produce enough power in that gear.
Downsizing an engine is another matter. I used to be under the impression that downsizing an engine was always better. However, if I needed to produce 50HP, a 2.0L engine (all else being equal) would almost certainly do so more efficiently than my 1.0L engine,
as long as it is geared correctly.