RCA inputs are generally 1 volt peak to peak, an amplifier output is generally 12 - 24 volts peak to peak when turned fully up.
If you have connected the DSP sub outputs (i.e. the speaker connectors) directly to the RCA connectors then you are probably over driving the amplifiers input stage and causing distortion and clipping.
Do you have a "high level input" on your amp? If so, connect the DSP sub speaker wires to that.

__________________
'02 X5 3.0d, Auto, Topaz Blue