So after spending way too much time Googling this issue, I finally found someone that has done this without taking the valve cover off - but they did remove the entire turbo and came in from underneath.
BMWxMAX from Russia has a very good series of blogs documenting all manner of maint tasks and mods.
https://www.drive2.com/r/bmw/1289884/
One of his blogs shows the entire manifold replacement...
https://www.drive2.com/l/467740493866860758/
So here's the thing... looking at his images and at my car, I really do reckon that it IS possible to replace the manifold without removing either the turbo or the valve cover.
So, borrowing a couple of BMWxMAX's photos, here's the above view, with all the covers off and the washer bottle removed: -
I will take mine to this point but also remove the vacuum reservoir, the turbo compressor outlet pipe (fat silver pipe at lower centre) and of course, the turbo inlet pipework (fat black pipe at lower right-of-centre). But that A/C pipe is going to get sworn at a fair bit...
And here's what it looks like from underneath: -
Old manifold is in place in the first image, new manifold in place in the last one.
This shows that even with the turbo removed, you cannot easily see or get to the top manifold bolts.
But more importantly, you can see the clearance from above, past the air filter housing. Plenty of room for tools and hands.
There is no argument - it will be easier to do this with either the valve cover or the turbo removed but given the extra work required (not to mention the extra cost in gaskets etc.) I'm not going down either path unless I have to.
A good magnetic mirror and light source are going to be a must however ....
Thanks to BMWxMAX for all the inspiration!