The V8 is visibly faster. It pulls strong on the highway, too.
But it eats tremendous amounts of gas. Even if you don't have a heavy gas foot, you will still get worse economy than even BMW posts.
The other topic to consider is that 4.4i X5s have much more standard features than the 3.0i. At least in Canada, that's the case. So you should research what features you are getting, aside from the raw engine itself.
I had a 2002 330i before this, which used the same 3L I6 used in the X5s. It is the BEST 6-pot I have ever seen, or heard, or driven. PERIOD. It's incredibly smooth.
EDIT: Now that I have a 4.4i, I would not go back to a 3.0i X5 since I'm used to the V8 power. However, had I been shopping from scratch, the 3.0i would have been more than satisfying. Also had BMW started to offer the new 255hp I6 in the X5, I could switch readily. It would only be slightly slower, but much more efficient. Alas, they will package that with the new-gen.