I have the same vehicle. 2008 4.8i, sport package w/ 19" wheels.

I have driven 2,000 or so miles at 'highway' speed (65 to 80 mph) under varying wind & weather conditions & have never experienced what you describe. My X-5 is 'rock solid' & feels planted to the road surface.

I think you need to have yours checked out by your service department, sounds like something is wrong to me!

__________________
2008 4.8i Mineral Green Metallic Sports Package, Technology Package, Cold Weather Package, Rear Climate Package, Premium Package, Premium Sound with iPod adaptor & Sirius Radio, Auto Tailgate, Head up display