I had the clicking sound on my 2003 Altima when pulling in or out from stop.
It was due to plastic part in the gear lever assembly worn out. When vehicle came to a stop, the plastic part would disengage to unblock and allow the gear to be placed to "Park". With time, it worn out and became making the clicking sound during engaging and disengaging.

But, I dont know how that would be related to X5 with their joystick assembly. Maybe there is something similar?