All the other joints are OK. The knocking is definitely in the rack or inner tie rods. I happened across a YouTube video where a guy with an E53 has exactly the same symptoms and he tightened the preloaded bolt by 15 degrees and that shut it up so I'm going to give that a try first later as it doesn't cost me anything..
If that doesn't do anything then I'm going to buy new inner and outer tie rods and church them on. If the roads here in the UK weren't so bad and full of holes I would never have noticed..