I had the same issue and it turned out to be a bushing in the lower control arm on the rear drivers side. Dealer MSRP was 314 but I found a new one on ebay for 130 shipped.

not sure if this is your issue or not

__________________
Previous: 2002 X5 3.0 / 121,000 miles
Current: 2007 X5 3.0si / 93,000 miles