Sounds like the engine is already pretty efficient in four cylinder mode, and that it's the acceleration (v8 mode, or is it s v6?) that knocks down your fuel economy. It was similar in the Sienna minivan my family had - could easily get 30+ on the highway, but around town it was easy to get low 20s unless you were careful.
Coasting to stops and avoiding hard acceleration will probably yield you the greatest improvement in fuel economy.
That being said, one simple yet effective mod is to bump up the air pressure in your tires to sidewall max (usually 44 or 51 PSI).
Regarding the factory air dam, that is most likely there to improve the aerodynamics of your vehicle by directing air away from the aerodynamically dirty underside of your car.
Side mirrors may yield a small improvement, but make sure that it's legal in your area to remove them. Many areas require at least a driver's side mirror in addition to your rearview mirror, and most areas require two outside mirrors if rearward visibility is obstructed (i.e., window tint, luggage, towing, etc.). If you do delete your mirrors, I would highly recommend getting a panoramic rearview mirror, such as
this one, and do some local driving to familarize yourself with driving without side mirrors - it can be daunting in a larger vehicle or one with major blind spots.
The
65+ efficiency mods link at the top of each page on here has several good ideas for improving fuel economy, from smooth wheel covers and underbody paneling, to more radical solutions such as wheel skirts, kammbacks, and boattails. It all depends on how radical you want to go, but always remember that 80%-90% of your fuel economy potential comes from , adjusting your own driving style (
100+ hypermiling tips).
~ Happy hypermiling.