xing6666, I feel your pain. We are all beta testers for the Chinese aftermarket systems. I don't care which system it is, the support is just lacking from what we have all become accustomed to. That being known, we all rely on each other to share our experiences and blunders.
On our X5 systems, most of us with factory navigation that had to run the 6M extension to the rear simply ran it down the right side under the kick panels but there have been also members that have taken the time to disassemble the entire interior and run the wiring along the factory harnessing. The time it takes is consuming and best estimate on time was around 8 hours but it is done right.
The RCA cable you mentioned sounds like the cable to the rear back up camera if you plan to install one. The red wire exposed on each end is not necessary for installation though some of us have used it for a power source as I did. We can cover rear back up cam installation if you plan on mounting one.
I have never done any work on the E38 but I believe you will need the iBus adapter and app to run the factory DSP, hopefully you bought that with your AVIN Avant-4. The app does support your E38 and a link to the developer is
https://www.ibus-app.de/
I will PM you my phone number and I can try to walk you through any questions you have.