I agree with agrawat. Google can't banned any site for a mere link exchange. Instead if your site had crossed google's terms and conditions, for example you have linked in a link farm, google will definitely penalize you.
I hate Link exchanges anyways most of it is spam - any type of exchange is really pointless - there is no real content