Yeah, that's why it is risky exchanging links or having reciprocal links with those sites. You need to have tools and knowledge on how you can detect those things.
links usually don't hurt you because of ONE FACTOR. competition. how easy would it be to link spam your competition and just knock them down via penalties in the SERPS.
Here you must note that you are talking about inbound links, outbound links can hurt your rankings or reputation.