You should keep your M3, like Greenkirby21 said, if you don't mind extra effort to get the kids back there.
I drive my ISF way more than X5M, even when going out with wife and 2 kids.
06' Infiniti G35 Coupe Copper 6 Speed Supercharge 498HP Full Load
14' Nissan GTR
12' BMW X5M