We have a few threads about this floating around the Google and SEO section plus i've provided a few articles in the SEO tip of the day sticky about fixing problems that led to a penalty.
What's your author text in your author box, if you are using same author text, it would be hit by google penguin as over seo. Tried to get building some profile link from authority site, and use "domain name" "click here" etc to change your backlink structure.