I have started to go through my set of nearly 18,000 distinct domain names that I link to from my site to remove the ones that are blacklisted according to whois information. I would be more interested in determining exactly what makes Google decide to not index a page for linking to SPAM or blacklisted sites. 1) Should this even be done (with respect to being closer to 100% indexed by Google)? 2) When a site is blacklisted according to whois lookup, is that essentially the same thing as a SPAM site - or are there cases where Google doesn't care when linking to a site blacklisted for a different reason? 3) Is there a good way to look up a domain thru Google (their API perhaps) to determine whether or not the domain is blacklisted according to Google and shouldn't be linked to?
Im looking for the same thing, i need to know what domains are or have been banned/ blacklisted. Sahel
That would be a great tool if it could check all the links on your site for banned sites.. Come on techies invent one!!!
The "Banned Website Checking Tool" that max pain refers to answers a few of my questions.... I think that that program would choke on my website... way too many links - it would get bogged down (my web has over 1,000,000 pages). I am more interested in checking a specific list of links - not ALL of my links. I don't want to remove more links than I have to. I like my content too much to throw a lot of it away - kinda like a pack rat in that sense. I am working on a program that takes my list of domains - and creates from that list a set of blacklisted domains. The program also will let you construct SQL commands so that you can effect data that refers to the blacklisted set of domains. I am still looking into other options for the program (integrating a XML API that returns whois info per domain - rather than web scraping). If anybody is interested in my program, PM me - it is still in BETA, and I could use a couple testers. I am still a bit uncertain what rule Google goes by... if a site is blacklisted, you shouldn't link to it - regardless of the blacklist status? Blacklisted is blacklisted is blacklisted.
The whois Db doesn't contain any information that can be used to determine if a site is "blacklisted." If you're not sure about the site you should add the rel="nofollow" attrubute to the anchor tag.
Under the Server Data section, for most pages - I can see a line like the following for domains that are blacklisted: Blacklist Status: Currently Listed For an example, http://whois.domaintools.com/duplibase.com Also, I would think that if my site contains any links that are blacklisted - whether or not there is the rel="nofollow" attribute, the page that contains that link will rank low - or not be indexed at all. Is my logic bad here, or does the "nofollow" avoid all problems with linking to blacklisted URLs?
I just want to confirm that after a bit more research on the subject, the addition of rel="nofollow" to any link should avoid any contribution to PR or any effect. I guess that my question now is - whether or not any page of mine that I know displays a blacklisted link - should I just keep that page from getting indexed at all, or will the page get indexed and treated the same as a similar page on my web that has no blacklisted domains listed.