it's not the blower or fuse because things are working fine after a few hours. Can't figure out what triggers this weird issue but it only happens when it's too hot (duh you need ac when it's hot)

Is it possible for the FSR to partially work/fail? On my e46 when the FSR failed it's just dead silent, no blower activity/noise at all. In my x5 when this happens the vents are just not blowing air (as if the vents are closed) yet the blower still runs fine and changes when I adjust the speed. The only vent that works is in the rear.