Well, one diode was burned out and was causing problems. You can see on the picture how dark it is.
Took it out and instead used a couple of diodes that I removed from a headlamp.
Soldered them in and now everything is working!