Currently, I think the issue was the door lock tumbler pin being broken.

That part that goes from the Driver door lock into the door actuator came out in 2 pieces. That pin with the round 4 point end was separated from the lock tumbler. I found the locking U shaped nut that holds that longer pin together split in half in the lower part of the door well.

One other thing I noticed before taking the door apart was the manual door lock was stuck and you couldn't press it down to lock the door.

I have ordered a door lock cylinder repair kit, and we'll see if that fixes the issue.