I think you are seeing a placebo effect.
I've taken mine off, though, for other reasons. I got tired of replacing wiper blades that have never or rarely been used so now I keep the arms and all in the cabin because on my cars they can be R&R'd with no tools. I haven't done a thing to the shafts. If by chance I do hit rain I only put the driver's side on, or I'll put them both on if the passenger wants it. If anything the delete reduces load on the mechanism but it doesn't noticeably speed up.
P.S. I used to replace them to prevent the famous arcing gouge in the glass from bad blades. It's the UV that kills them. I don't remember them going bad so fast and so often back in the old days.
