In addition to the good cautions noted above (limited ground clearance, limited wheel travel, lack of underbody protection, street tires that can't handle mud) I would add two points. Firstly, the X5 doesn't have a frame, it is a unibody. It isn't built to handle uneven terrain. Secondly, what will often strand you isn't mechanical, but rather electrical. Many of the control modules and wiring harnesses are mounted low in the chassis, subject to water ingress. You mention crossing streams, but be cautious. Not only will water stop you, but it isn't the kind of thing you can fix when out in the bush.

The X5 is essentially a taller e39 5 series wagon, with awd. It will arguably go fewer places than a 5 series wagon, since it is heavier, and has a higher centre of gravity, while having very similar ground clearance. The awd can certainly help it go more places with the right tires, but I always remember a line from a shop I worked in many years ago. There, we ran a 4x4 tow truck with an additional front winch, for off road recovery. We described awd as just a tool to get people further down a road that they shouldn't have been on in the first place. Because people drove in with awd, if they got stuck they couldn't get out. If you drive a 4wd truck off road in 2wd, the 4wd lever (plus your winch) is your exit strategy. I just remember that awd and dreams of off roading were great for our business.

__________________
2007 X3 3.0si, 6 MT, Premium, White

Retired:
2008 535i, 6 MT, M Sport, Premium, Space Grey
2003 X5 3.0 Steptronic, Premium, Titanium Silver
2002 325xi 5 MT, Steel Grey
2004 Z4 3.0 Premium, Sport, SMG, Maldives Blue