The X isn't designed to be an off road vehicle (althought the X3 has done very well in off-road racing). It's designed to be a performance on-road vehicle with improved utility over a car. Even with a drive train suited for off-roading, the X5 ground clearance is still too low to be very usable off-road. The X3 has better clearance, but both are still far behind other vehicles that are designed to be driven both on and off-road.

In my opinion, keep it on the road where it belongs. If you have a need for an off-roading vehicle, buy something else.