It's hard to say. It could be so many things. I actually have the same problem and just decided to live with it. I didn't want to spend the time & money to look into the issue. If you were to do a search on the boards regarding this, every one has a different solution, if any.
How many miles do you have on your X5? What major repairs have you done lately?
Here's a thread that's relevant:
http://www.xoutpost.com/bmw-sav-foru...t-brake-2.html
If you do get it fixed, be sure to let the board know. Good luck