More info:
I should have said in my first post E53, 4.4, '04, 28k miles. Well, I guess the first is implied by the forum
I left the blower fuse out for a few hours and when I inserted it, before starting the X5, the blower fan didn't turn on. I checked the voltage and charge with my slow charger before cranking the engine and the values were 12.7V and 95% respectively. I don't know how accurate the device is but it was 12.7V and 100% a few hours before (immediately after I stopped the engine). IIRC, my OEM battery used to read 13.5V on this device.
After I started the car it didn't turn on until I turned on the AC and fan from the console.
While the engine is on, if I turn the console fan control totally off (screen goes black), no air comes out of the vents but the blower fan still blows loudly (this fan noise is coming from a different location than the vent fans, it sounds like from the right of the engine bay).
When I shut the engine off, the blower steps down a little but is still very noticeably on (the noise appears to be coming from the passenger side under the hood). However, no air is coming out of the vents. I left the fuse in while I went into a store for about 10-15 minutes and it was still on when I returned. When I got home, I removed the fuse so the blower wouldn't continue to run.