Here is the X5 rear rear wiper arm parts diagram:
Here is the X3 rear wiper arm parts diagram:
I have not taken my X5 rear wiper off but from what I have seen, it's held on by a nut. I would think that if the X3 wiper arm is held the same way you could swap them out. The main issue then would be the length of the arm from the X3 being long enough for the X5.
Good luck!