I was exactly in the same situation but only because ordering the M3 Sedan meant I would have a car by now (July). If you have any consideration for your family's needs, get the X5. I seriously thought about getting the sedan for the 'practicality of a four door', but everyone knows an M3 is far from 'practical' (and it's best served as a coupe).

On another note, you may be surprised at the responses you get at the M3 forums. Most of the folks will tell you to get the M3 and get another car, but then you'll be back at square 1. You'll have a rarely driven automobile and another car to compensate for day-to-day needs. Most of the other folks will see "family" and think X5.