Happens to me too Vinuneuro. Is it a big drop in dl speed? How many files are being downloaded and what is your total dl speed?
For every packet you receive your BTclient has to send out an "OK got it" before the next packet is sent out. So if you are using up most of your bandwidth on the dl side and then browsing around using up the rest of it you'll end up in a never ending queue with your torrent client constantly waiting for a ping! slowing the whole process down.
Hope this answers your question.