If you have a link exchange going, make sure that you check it every month or so to make sure that you are not linking to a bad neighbourhood. go to http://www.iwebtool.com/visual_pagerank and see if all of your links have a PR. If they do not, go to Google and make sure they are indexed (use site:www.sitename.com.au). If they are not indexed, they may be incurring a penalty, and you may also get one as well for linking to them. Remove them immediately to avoid any penalties. You will find that SERPs increase as soon as you rid yourself of the bad eggs. Also, check that all the links you give out are relevant to your visitors. Useless link exchanges can be dangerous, and may give you large penalties if you link to the wrong people. Make your site for your visitors and not for SEs (build it well and they will come)