I don't think the M3 feels fast at all.. It feels SLOW in comparison to my remapped 335d.
It does not have that shove of a turbo.. But it does actually perform. It puts down the times. It's quicker than what it feels.
My 335d didn't sound loud at all. It felt properly fast though because of the torque.
I have had both 335 (ok D, but have driven plenty of i's) and E46. Both great cars.
Thing is mate, it's harder to extract the M3's performance than the 335i. In gear the 335i is pretty monstrous. The M3 would need to be on the right gear and ready for it.
Standard for standard the E46 is quite a bit quicker, but get a average driver inthe M3 who does not drop that extra gear or hang on for that last few 100rpm and it then becomes a slow car (relatively speaking) in which a 335i will beat it. Because the 335i has so much torque to lean on and does not need dropping to 2nd at 50mph from 6th gear like the M3 does!
.... But I now have a M3 CS