With the help of some info from Steved on the bimmerfest forum, I printed out his posts and took them and my car to the local dealer.

They replaced the central control unit and I haven't had any problems since.

__________________
PREVIOUS

2006 E53 X5 3.0D
2004 E53 X5 4.8is
2004 E53 X5 4.4 Sport
2011 E70 X5 3.0D with Aero kit
2009 E71 X6 50i Sport
2009 E70 X5 50i Sport
2007 E60 540 M Sport LCI (All time favourite car)
2007 E70 X5 4.8i Sport
2005 E53 4.8is
2003 E53 3.0 Sport