I had the exact same problem a few weeks ago and repaired it myself, based on some feedback from a few people on this forum. Mirror would go left/right but not up/down.

FYI, there is only one motor for all 4 directions.

Here is a video on removing the cover and the mirror glass. You only care about the first 35-40 seconds or so... The rest you don't need to worry about.
001 BMW X5 HCE-C500 TOPVIEW camera installation - YouTube

After you get the glass off, its only 4 screws and you can remove the motor and replace it.

Here is where I bought mine ($170 including shipping).
BMW 67136973724 Genuine Factory Original Mirror Motor | eBay

Be sure this is the correct motor for your X, here:
RealOEM.com * Online BMW Parts Catalog

Whole job took about 15 minutes.

__________________

2008 4.8i X5 (2/25/08) 105k:
Titanium Silver/Black; All -HUD; Aero Kit; 22" VS-130 w/Pirellis;
///M Shifter & Fr Brakes; Gauge Rings; Console Pads; Custom Wheel & Mats; Roof Rails
Carbon Fiber: Int Trim, Nav Surround, Grill, Cluster Trim, Shifter & Eyelids
1.2" H&R; LUX AEs; LCI Tails; Rear Fogs; NBT EVO; ESS Tune; Meisterchaft GTC Exhaust