"But isn't the added mechanical advantage of higher compression counteracted by increased energy loss during the compression stroke? It seems to me that if it takes more energy to compress the fuel/air charge further, but you get more energy out of it, shouldn't the two factors just cancel each other out?"
Air is a perfect spring. 100% of the energy used to compress it is directly returned when it reexpands, at least thats how it would be in a perfect world.
The reality is that higher compression creates more piston friction, and part of the heat of compression and therefore its energy is lost to the cylinder. If this was'nt true, youd'e have a perpetual motion machine.
The way it works out though, a large % of the energy needed for compression is in fact simply returned to the piston on the downward stroke, and the extra friction and heat losses from bumping compression from say 9:1 to 11:1 is a fairly small increase, and far offset by the extra mechanical advantage gained by the increase.
This is especially true when an engine is throttled way back in cruise operation. The net effective cylinder pressure is reduced by the throttling, hurting system efficiency. An engine can actually handle much more compression at low throttle settings without knocking, but has compression ratio's chosen to handle heavy throttle application to optimize max output.
I've often wondered what sort of mpg results you could get by going sky high on compression to max out efficiency at cruise settings and just greatly retard timing for heavy throttle, ( making efficiency and max power at heavy throttle suffer) since the engine really spends very little time at heavy throttle anyway.
Dean in Milwaukee