Otto,
Small wheels have a higher rolling resistance - at least traditionally... So that's not a good thing... Plus, the "big boys" have fully faired front drive wheels - only a tiny bit is actually exposed. Something like this
I mean, it's probably worth a look
There very well could be a point where it would pay off.
As far as fitting under the fairing... Your leg path tends to be the limiting factor, not so much the wheels... At least, traditionally.
Also, as always to point it out... Sailplanes are generally are bigger - and something tells me faster than a practical application (perhaps closer to a world speed record velocity though).