As an electronics engineer, there can be a number of causes for this type of problem.
The most common cause is usually the light socket itself, but it can also be caused by a bad relay, or bad wiring/connectors to the light socket, or a bad/intermittent earth.
I’m not sure how you connected in the HID units but obviously the fault was still present during that time. If you used a separate relay as you mention, you can probably discount the original relay.
Was the original light socket used when the HID units were installed? If not, this would tend to point towards a bad/intermittent earth. If it was used I would point toward this as the most likely culprit.
These sorts of problems can be very difficult to find, particularly as intermittent as this one seems.
Sometimes it is quicker, and cheaper if you’re having to pay somebody to do the work, just to replace all the possible components in one go.