I removed my muffler on my TDI as well, and added a straight pipe:
The turbo already reduces the sound a great deal on its own... without the muffler, I can hear more of the turbo whistle and a little more rumble, but a casual observer (not familiar with TDIs) wouldn't notice the lack of muffler. I didn't remove the muffler for FE or power purposes; I removed it because I really wanted to hear more of the turbo. Ya, I know, soft in the head & all that...