It's P/N:32306778412 if you want to replace it OEM.
If you want to remove the trim piece you have to remove the airbag and then there are about 5 torx screws you have to remove.
Check this out. X555a did this, removed the trim piece, sanded it down, sprayed it and then lacquered it.
BMW X5 • View topic - replacing steering wheel trim e53
Also see thread about removing trim piece:
http://www.xoutpost.com/x5-e53-forum/...ing-wheel.html