Toe in alignment will hide some problems. I had basically the same symptom you describe and was the steering rack. I still have the replacement rack from 75,000 mile car. Never installed becusse deer killed the car. We can look up shipping cost If it'll be cheaper than sourcing locally.

You can confirm steering rack slop by having somebody steer left and right on the ground while holding the rack it should make an obvious clunk as it wobbles bweten the backlash.

__________________
2011 E70 • N55 (me)
2012 E70 • N63 (wife)