This is interesting, I had a similar problem, but mine reduced the car into a complete running mess. I swear if it was a SI engine, I would have been looking for a cylinder failure on the spark.
After an eternity of testing everything, the last resort was the fuel filter, before the HP pump. It ran like new again, well at least for half a dozen months until the exact same symptoms presented once more.
I know that we get the shittiest diesel in the world here in the underverse and they even propose that there is such a thing as premium diesel and regular diesel. This was factually debunked and the only real benefit found was it foamed less at filling.
Anyhow, back to what causes these FUBAR readings and behavior. I also had the scanner reading of all glow plugs / injectors are faulty These were over six years old as were most of the faults listed, so much for workshop services. It was reset and I started searching for replacements. Here is where I got lucky...My uncle who used to run the RAA (road side assist) in the south east of our state, now has been the Euro Car specialist of the South East for over ten years. No, I didn't get out to see him much for my own health reasons.
He checked my car 11/2006 E53 X5 and found that all these were operating fine. In fact, after all the issues I had fixed, he was not surprised at all to hear that when I put the oil flush through the engine and drove 14 km, it was starting to improve. Whilst he does not advocate the removal of the EGR valve, he does say it should be regularly checked and cleaned if it is still fitted to the vehicle. Here removal can cost you a huge fine, if caught.
Due to this clogging and EGR failure, I have had my car shut down completely as I stop at a set of lights, have idle hunting so severe while in gear that the car will stall and in neutral see 200-300 variation on the tacho. The more clogged it gets the more it takes to start when hot and yet when cold or sitting idle for a day or two, it starts at first crank and idles so smooth you don't even notice it is a diesel.
These are my own personal adventures with this mystery of the cranking and then what I call the diesel misfire. I have only experienced what you have once, and it was filter related...all that came out was what looked like fine metal rust particles. Well we know that is not the tank, don't we
Right now, I keep blowing turbo hoses off and without that working, that 3.0ltr motor is useless moving that lump of metal.
