Hi all....I realize this is an old thread but it was very useful, so thanks to all for posting. Because of this, I decided to post my experience. My mirror was the driver side. It folded in but not out. I had to push it out with my hand and even then it would not stay out. It would swing on acceleration/breaking.
This is how I knew the motor was good. So I opened it up to check if the gears were stripped and the were not.

I used this to open the mirror.

https://www.youtube.com/watch?v=ZsxDsxR90UM

Bought the motor housing from here.

Bmw X5 E53 Wing Mirror Folding Repair Replacement Motor Housing

I only needed the outer housing (left in picture).

Thank you to Motorsport1207 for the excellent description and pictures.

Here's how I did it.

I started by removing the mirror back housing piece as in the link to the YouTube video. Thanks to frenchybetter for that post.

There are 4 torx screws that hold the mirror on the top of the elbow hinge, remove those.

Remove the bottom cover where the mirror hinges. This will expose 4 more torx screws, remove those. Take notice of the positioning of the motor before removal. This is important as it will change the way the mirror goes back on.

The motor will then be accessible and will expose 4 more torx screws. Remove those last 4 torx screws and replace the housing.

My housing was cracked and had the pins broken. The pieces of the pins were still in the bottom part of the housing. I had to pry those out with a very small flat screw driver and needle nose pliers for removal.

Remove the white ring on the top of the damaged housing and place exactly with the same positioning on the new housing.

Reassemble in reverse order.

Total time took less than an hour and less than $30 in parts.

Mirror is good as new and needless to say the wife is happy!

Thank you all for sharing.