E53 only starts on weak battery when alternator shot

[ala2000]

I have a 2001 bmw x5 4.4i. The car battery icon on dash light up and I checked battery is at 11.9 at idle so the alternator is shot for sure. I happen to have a fully charged battery, I swap it out the battery and try to start the car. As soon as I turn the key to start position, I hear power down noise (do not know how to describ). everything went out. No lights, cannot open trunk lid. I check the battery bolts/terminal at front, they are 2.4 v. I manage to open the trunk lid and metered battery which is 12.2 v inside trunk (initially is fully charged should be around 13 v). I disconnect the battery and reconnect it. At 12.2v, I can start car right up, everything seems to be normal. (that was yesterday)

Today, I jumped the battery to 12.6 v via my other good car. Above situation happen all over again. Now I know better, I leave the truck lid open when I 1st start car. After initial fail, the battery immediately went down to 12.2 and I disconnect and reconnect battery, everything seems normal again except the battery light on dash. Also at idle, the battery is about 11.9v.

What to do right now. I try not to put new alternator on before I understand what is going here. Thanks in advance.