I have a 2007 335i (N54), and recently added a 2011 M3. Both are E90 models, however the M is a 6MT, whereas the 335i is an auto.
For those that complain about the lack of torque in an M3, I really do recommend that they be driven back to back for an honest comparison. My M3 isn't even broken in yet, and I find the torque to be on par with the N54 even whilst keeping the RPMs below 5000. Additionally, the M3 is renowned as one of BMW's most reliable models in recent times, whereas the 335i has it's fair share of documented quirks on this forum and elsewhere.
If you are fortunate enough to be able to keep both vehicles, I whole heartedly encourage you to spring for the E90 M3, especially since it will likely be the last M car in a naturally aspirated guise. The sound of the V8 is priceless!