Start with the rotational mass, if you are willing to dump money into the project (though, read on below for plenty of cheaper/free weight reduction tips!). Light weight wheels are a perfect start! I used to buy the lightest wheels for the buck I could, which was the Rota Slipstreams at the time. The difference between these and heavy rims, for acceleration, was very noticeable. City driving, it should net you a little MPG, though most aluminum wheels will come with challenges in the aerodynamic department.
Another one that kills weight and adds "umph" is aluminum drive pulley. I'm sure there is a little MPG to gain, as it takes weight directly off the engine's rotational mass, but probably not a lot. An under-drive pulley sends less power to the accessories, giving even more reving hapiness, likely giving you further MPG gains, but each car responds to this differently and you will want to do your research before going this route.
Ridding of sound deadener from the factory and using carefully placed, higher quality deadener will yield in about the same results, but save you ~20-30 lbs pretty easily. Carpet padding is another thing you can take out, but will gain a tad bit of road noise. You can usually find thick insulation on the firewall inside the cab that is pretty heavy as well.
Your best bet is basically remove as much interior as you are willing to remove, analyze it, and put back what you believe you need. You'd be surprised how many little things are in there that you may think you don't need. e.g. my Insight had a metal plate in the steering column cover that I saw unnecessary, so I yanked it out. Why lug around even an ounce of weight if it does nothing for you? Sure, maybe some of it is there for a reason, but if those reasons don't fit with your agenda, YANK IT OUT! Course, be very sure of what you pull out. You don't want to compromise your safety too much, now do ya? (though, airbags are heavy... are they worth your life?)
I like to find the small things that I don't need...
here's a spreadsheet to my insight, for example. In my old Neon, I was willing to cut sheet metal and bracing out to save weight. Hell, I took bolts I deemed too long and cut them down! Or replaced them with zipties, where deemed safe to do so...
My Insight, I'm a little more reserved about making permanent and extreme changes. Every car is different, but has a lot in common with the same weight-reducing capabilities. All depends on how extreme you are willing to go!
Just an FYI, if you aren't primarily city driving, weight reduction is going to net you close to nothing in MPG. Sacraficing weight to get aerodynamic gains is definitely the right way to go for the highway! Examples: wipers, mirrors, antennas, badges, "rub strips" on the doors, emblems, etc.
When adding weight in, consider using aluminum and coroplast. Great for things like grill blocks that, even in city, are very well worth the tiny amount of weight.