If you are using an NS that is blacklisted/etc, on your domain(s), would it affect your ranking or any other aspect of your site?
I'd be surprised if they blacklisted the actual nameserver. It wouldn't make sense for the search engines to do that.