Ok, so I swapped out the relay, the washer motors worked for a second then stopped. It didn't stop me though. Here's what I did.

FIRST, Drain the reservoir if it has cleaner in it! I used a hose and siphon.

I know that DC motors are pretty hard to kill. To my happiness these washer motors come apart easily. If I would have broken one taking it apart, it was already broken... And they aren't very expensive.
At least the front and rear washer motors come apart easily. The headlight is a little larger and stubborn I didn't bother taking it apart. It passed the bench test though.

For the smaller two, with the motors removed from the reservoir, at the workbench, just pull firmly and the top will come off. **Be sure to pull straight up and don't twist, or you might twist off the contacts.** (I bent them on the first one)

Now with the top off and the tiny DC motor exposed, just pull it straight up and the shaft slides out of the pump mechanism. There should be a rubber shaft seal either still on the motor shaft or inside the pump mech.
Clean that out really well too. A little brake lube or Vaseline like paste will help keep the seal when you are putting it back together.

I cleaned out the rust and shot them with a little PB Blaster (or WD40).
I gave the shaft a couple back and forth turns with some pliers, then I could spin it with my fingers... Blew it all out with a compressor, and tested them with a 12v battery on the bench. The motors work great now. Put it all back together and it is working. Put the old relay back in. It is working fine too!

On another note, I don't know why I didn't get voltage when I checked the plugs, but it must have been a user error. :P

Hope this might help someone else too!

Thanks Xoutpost! wouldn't have been brave enough to try if I didn't stop here. Also Thanks to Qsilver7 and smokeyyank for the info!