Google checks some sites manually. JhonChow was ranking first for "make money online" for a lot of time because of a lot of people reviewing him with this anchor text. They dropped him to position 60-70. On the other hand I don't think it is possible to get bad rankings for poor backlinks. Google won't penalize you for any kind of backlink you get because you can't control this. Why should they penalize your website because something you can't control. That's another reason why you will never get penalized if you get some backlinks from link farms (even if they are worthless, they can't harm you).
Google has admitted that Google Bowling does exist: http://www.forbes.com/2007/06/28/negative-search-google-tech-ebiz-cx_ag_0628seo.html
I would say that it' still based on where the links are coming from. Google is smart enough to know which links are more valid than others. So if your keyword (one anchor text) SEO is based on backlinks from poor sites, I am sure they won't let the PR points pass. The opposite of G Bowling, Sandbox also doesn't apply to all sites. When Microsoft took the #1 spot for Zune, they were rapidly gaining backlinks, appeared in #6 or 7 and took the first spot in few weeks. The domain was bought some time ago (possibly the original creator of the domain was someone else as zune.com is not owned by Microsoft either). And for a site to rank top 5 in few weeks, you would need massive backlinks. But zune.net didn't go through sandbox regardless of its theory. So, likewise, I believe G does things that we can't measure. It's very possible that they do things manually, like Glo-con said. If Glo-con's website is reviewed by G staff manually, then I guess his site could be penalized by one-anchor-text backlinks.
Why should my site get penalized?? I'm using 10-20 different anchor texts and I often link deep inside the Directory instead of linking always to the homepage....
No I don't think yours should get penalized even if using one anchor text to build backlinks unless G staff checks your site manually and monitor each move + they are in a bad mood But using different keyword combination or even keyphrases are better. I use 7 or 8 myself.