No need. If you were going to go up to a 4.6is or 4.8is then it would be worth it. Otherwise, I say be happy with what you got. Unless, you are wanting something newer to avoid repair costs.

From Edmunds;

What's New for 2004

The high-performance X5 4.6is is discontinued. The 3.0i and 4.4i receive an exterior update, as well as a new all-wheel-drive system called xDrive. A six-speed manual transmission is now standard on the 3.0i, and a six-speed automatic is standard on the 4.4i. The 4.4i also receives a new 4.4-liter engine with Valvetronic and a fully variable intake system. Both models can add Sirius Satellite Radio and Bluetooth hands-free cell phone capability as dealer-installed accessories.

Plus, everything I have read says that the pre-facelift 4.4's pull like beasts off the line and the 04's+ have a slightly different engine feel.