On mine, it was a vacuum leak at about 170,000 miles. The intake boot had been replaced at 156,000 miles. But this time, all the other vacuum lines were replaced. That bumped the idle speed back up to 650 RPM where it belonged.

2002 X5 3.0 242,000 miles
2004 325i 107,000 miles