I've read about diesel engines that can achieve up to 6,000RPM, so higher RPM in a diesel isn't impossible. I'd suspect the real reason is it goes well with it's instant torque characteristics. There's no wating for inrushing air to fill up the vacuum created by a throttle plate that is now open.
But generally lower RPM does help reduce friction, also helps keep the throttle open and can help make up for less than ideal induction and exhaust.
I drove a Chevy gas van the other day that would shift up and place the engine at 1,100RPM if I didn't put my foot into it.
__________________
|