Even though a Diesel engine above 4000cc could get the very same or occasionally some better mileage than a smaller one due to the differences in RPM bands and low-end torque, such a restriction based exclusively on the displacement is likely to become detrimental to the fuel-efficiency and eventually the performance in rougher terrain.
Let's consider, for example, a Toyota Hilux. Even if it would be possible to shoehorn a 4.1L 15B engine into it instead of the 3.0L 5L-E, the 2.5L 2KD-FTV, the 2.8L 1GD-FTV or the 2.4L 2GD-FTV available in different markets around the world, a heavier engine is likely to increase the tendency for the vehicle to have its front-end "sinking" in soft, unstable terrain. The dead weight inherent to a heavier engine may also reflect in a decrease of the payload, thus decreasing even further the efficiency of the vehicle considering the specific fuel-consumption for the payload.