yeah, I was in the same position and yes the steering is tougher but it's safer to steer with both hands anyway so I decided to be happy with it.
I even went in to the dealership because my steering would not come back to normal position after turning at low speeds (around 5km/h during parking maneuvers). They made the steering a bit lighter (maybe put a bit more grease?) but made no drastic difference. I can definitely live with it for now but my next one will have active steering for sure.
After driving several other loaner 3 series BMWs I find that the X5 steering is the heaviest among the models.

I wonder why...