I've done it a few times on the X3, and it holds up surprisingly well (better than I'd expected). Oddly, the ride doesn't seem that bad when I took it on a narrow dirt trail thru a forest, although admittedly I wasn't going faster than about 10mph.

There was this memorable time when I drove it up a steep trail to get to a scenic viewpoint. Right when the hill got real steep, there was a huge embedded rock on the left half and loose gravel on the right side, and I was preparing for the worst. Interestingly enough, X3 had did it, no sweat.