Incoming links can't really hurt you. If they could, other people could do destructive links to their competitors and ruin their ranking. At worst, a link will do nothing for you. Outgoing links on your site can hurt you. If you have external links on your site that aren't really topic related, get rid of them or use rel="nofollow". This is more important for site wide links than for deep links and article links.
Register a domain today, get it indexed by next week with a couple of backlinks, and I'll kill it within days. Guaranteed. Truthfully, incoming links will only penalise your site if its a new site, usually you'll get sandboxed with several dodgy links in the beginning. Get quality, related backlinks in the beginning and you'll never see a sandbox again
If they are relevant quality links this will not happen; however if you just build a ton of directory, social bookmarking, irrelevant low quality links you'll be playing in the sandbox with kitty.