No not a direct swap the m3 seats have the seatbelt mounted to the bottom of the side pillar while the X5 seat belt mounts to the actual seat rail. You will need to swap the outermost seat bracket rail or use a 4dr sport seat form an e46.

__________________
04 BMW 3.0i X5 on 22"
04 M3 6spd