Hey Guys, As some of you may know, the problem of proxies scraping content which is then linked to is causing duplicate content in the search engines. !ask! has released a phpLD mod which allows you to prevent these proxies from scraping your content and details can be found here. However for those who do not wish to register at the other forum in order to download the script, there is an alternative method which is to simply block the IP address's by using the following code in the top of your .htaccess file (This can be applied to any site, not just directories) <Files *> Order Allow,Deny Allow from all Deny from 64.209.134.9 Deny from 66.90.73.236 Deny from 206.123.117.117 </Files> Code (markup): This will produce something like this: http://bhomiyo.com/en.xliterate/directoryshare.com Thus far, during the course of my research into this problem I have picked up all of the following IP's from proxies which have already managed to get cached copies of directories into the search engine results. 64.209.134.9 bhomiyo.com 66.90.73.236 schoolsurf.org 206.123.117.117 proxy.eu.com 206.123.117.115 boredsurf.info 207.234.209.59 freshtart.com 66.232.113.242 a1proxy.com 74.208.14.215 surfhot.info 216.32.94.194 proxytr.com 38.100.42.25 crazyproxy.info 72.232.177.45 concealme.com 67.159.35.59 proxydrop.com 66.226.27.21 workfriendly.net 82.165.244.16 ipzap.com 66.90.103.37 proxylobby.info 64.151.124.5 123download.info 123proxygod.info antiwork.info banover.info beanon.info beanybody.info beanyone.info beanywhere.info bestsupersearchsite.com bestwebsitefilter.com besurfing.com bewherever.info breaknow.info breakrules.info burnoutproxy.info cancerproxy.info cantstopit.info cantstopthis.info classsucks.info cutchains.info earlylunch.info facebookfilter.info filtermyspace.info finallyonline.info financialproxy.info geteverywhere.info getsomewhere.info getthrough.info goeverywhere.info gosomewhere.info hatework.info hideguy.info hidemyinternet.com imfreenow.info instantinvisible.info leetproxy.info mylinkladder.com mysitefilter.info myspacefilter.info myspacekid.info myspaceproxy123.info mysupersearchsite.com mywebsitefilter.com nerdproxy.info newsupersearchsite.com newwebsitefilter.com notbound.info opentobrowse.com secondperiod.info sitefilter.info sneakyinternet.com sneakysurfer.info soccerproxy.info sportsproxy.info springproxy.info stealthysurfers.com studentloanproxy.info summerproxy.info themyspacebypass.info themyspacekid.info theproxybypass.info theproxygod.info theproxyhost.com thesitefilter.info thesneakysurfer.info thesupersearchsite.com thewebsitefilter.com unblockmyspace123.info unblockwhat.info 66.90.104.77 proxyprince.com 64.85.160.59 surfindark.com 81.176.236.12 serjwizard.info Code (markup): Please, if you have the time add more proxies to this list which you have seen showing up in Google, Yahoo or MSN with cached copies of other peoples web sites.
Thanks Pete.... I also forgot to mention that this list of IP's can also be used with !ask!'s phpLD script based solution.
Thank You I respect and hold in high regard members who are helpful and provide information to help the directory community
The code given was just a short example, use all of the IP's found in the main list. I posted it seperately so as a group we can continue to add IP / Domain's to it.
Lol, I noticed one of my proxies is on the list. I think this is a semi-smart move, but there are a million (at least) proxies out there.
Well they'll be never a way to block them ALL, but its always better to block some than none. I was looking for something like this Silky. Thanks a whole lot for doing this. A tool like this is not only helpful to directories, but any other site you may run as well. Gj.
bobby, Prevention is better than cure, I am only interested in blocking proxies which are coming up with cached versions in the search engines, not out to black list all proxies. Leopard, your quite welcome
LMAO .... seeing this thread i made a small hack this afternoon! and guess what i got banned! first time im thankful for getting banned http://bhomiyo.com/en.xliterate/dir.vc was a simple javascript at work, some strip javascript....so wont work there i ran a for loop with all the names above with the function gethostbyname(); following are the IPs i get, (assumming this is the way to go about it!, never played with IPs b4)
I'm getting the same message from them and all I have done is block their IP Address. http://bhomiyo.com/en.xliterate/directoryshare.com
i made a small php code, similiar to ban IPs, putting a small js function to redirect to my site! anyways, anything that works, Thanx again Silky! M.
Great post and just to add to this here's an additional snippet for you to consider.... Full post here http://www.seofaststart.com/blog/google-proxy-hacking Hope it helps.
Ooops, thought I'd put that link in already And I agree, sadly though it's like Google telling people to use nofollow on paid links, it's something thats never going to catch on.
Thanks SilkySmooth!!! As already mentioned there is no end to block all proxies, but it's better than nothing. I guess Google should be worry about this issue and start working on it.
It's really sad anyone has to implement such protection. Why the fonk is Google not fixing the problem with proxies and SERPs?