Very simply, reviews increase conversion rates. And several studies point to the fact that, yes, even negative reviews help.... Read more: http://www.grokdotcom.com/2009/05/06/you-can’t-edit-your-way-out-of-negative-reviews/
Yes, by keeping the negative reviews (without insults) on your site you're able to add in bold character an answer from your company/yourself explaining your point of view. But if you simply delete the review, there are many other places a frustrated customer can post his negative review on, and there you'll have more trouble going and joining the website to answer the critics. And honestly it looks real fake when there's only positive reviews about your product on your website, can't be trusted!