Unfortunately you will not be able to replace it without removing the complete center console, and then disassembling it a bit more.
If you try to go your way you will probably break the cover.

It is not that bad, you need an afternoon and a guide that is floating around the forum and it can be done. I did it a few months ago.

__________________
2003 / 3.0i / Auto