A pox on spammers! 
They've basically used your e-mail addy as the return address for a mass mailing...
I gout hundreds of 'cannot be delivered' e-mails the other day - bastards!
And there is fuck all that you or your ISP can do about it! 
So, let me get this right. My existing clients
may now be receiving emails offering HD DVD's of anal with Margerita and my Active Marketing Programme might include offers of Viagra and penis enlargement to potential new clients? 
Yay, yay and thrice yay, I am truly cattle trucked 
Should I ditch my ISP? Will it be the same if I keep my domain name?

I would very much doubt it…
They will have their own mailing list that they have collected from various sources - including your own address prolly.
They just chose your address randomly as the return address to help cover their tracks.
So, it is unlikely that a large percentage of your clients will be on the list – if any.
And, as Snoopy says, it is common enough now that most people are aware of it.
The piss-boiling thing is that everybody says that nothing can be done about it because the e-mails are sent from abroad or often sent from other people’s systems using virus programs. However, it doesn’t take Einstein to realise that if they are promoting or selling something you only have to follow that link to the source…