I am in the process of starting this. I watched a video on youtube where this guy suggested that you could remove the 4 bolts holding the metal plate thing on the passenger side, then drill out the rivets and then have just enough room to change out the fuel filter. It seems easier than removing that entire cover. I have started on the some of the bolts and still have like 100 or so more to go (ok about 20..).
I like this idea, but those that have done the fuel filter will know if it will be too tight. This is for my 4.6is...
https://www.youtube.com/watch?v=bGj92asy9U8
I think at about 4:19 he mentions this approach in more detail.