I have had the same problem from day one with my X5.

The dealer swapped the battery over and it cured it for a while,but it still happens,turn the key to start and just a click,turn it off then back on and it starts.

I read somewhere that they have a fault with the ignition switch, I have bought a starter button kit complete with a relay..when I get round to fitting it.