My wife has an 07 328i and I bought an 05 X5 3.0 myself. While I don't own a 8 cyl. X5 and can't speak for the fun people here are having with them, I can surely speak for the size and weight of this vehicle. You will NOT get the 'driving' satisfaction that you are getting with your 335 from this vehicle.

If you were saying you will only have ONE car in your family, I would tell you that the 3.0 (while not a slouch) is lacking in the fun factor and would recommend you getting a 8 cyl. to have fun with. But since you already have a fun car, I would suggest going with what the wife wants/needs, and leaving the extra cash towards getting another fun car (similar to your 335) after your lease expires.

p.s.
When we were looking at BMWs back in 2002 for my wife, we went and checked out a Mercedes dealership (I am/was more of a Mercedes person than a BMW person) and I kinda convinced my wife and we ended up buying a Mercedes that she liked A LOT. But after a few months, she kept talking about how she should have gotten the 3 series and such. What I learned from that experience? From now on I will get my wife EXACTLY what she wants. That is what we did when getting her the 328... I would have loved to have a coupe, she wanted a 4 door... we got the 4 door, ... I kinda preferred the 335, she wanted a 328... etc. etc. Even if what she wants is inferior to what you are suggesting to her... later on you WILL hear about the fact that this is NOT what she wanted! (if she is anything like my wife, that is!