Protecton against Proxies

Hey Guys,

As some of you may know, the problem of proxies scraping content which is then linked to is causing duplicate content in the search engines.

!ask! has released a phpLD mod which allows you to prevent these proxies from scraping your content and details can be found here.

However for those who do not wish to register at the other forum in order to download the script, there is an alternative method which is to simply block the IP address's by using the following code in the top of your .htaccess file (This can be applied to any site, not just directories)

<Files *>
Order Allow,Deny
Allow from all
Deny from 64.209.134.9
Deny from 66.90.73.236
Deny from 206.123.117.117
</Files>

Code (markup):

This will produce something like this: http://bhomiyo.com/en.xliterate/directoryshare.com

Thus far, during the course of my research into this problem I have picked up all of the following IP's from proxies which have already managed to get cached copies of directories into the search engine results.

64.209.134.9    bhomiyo.com
66.90.73.236    schoolsurf.org
206.123.117.117 proxy.eu.com
206.123.117.115 boredsurf.info
207.234.209.59  freshtart.com
66.232.113.242  a1proxy.com
74.208.14.215   surfhot.info
216.32.94.194   proxytr.com
38.100.42.25    crazyproxy.info
72.232.177.45   concealme.com
67.159.35.59    proxydrop.com
66.226.27.21    workfriendly.net
82.165.244.16   ipzap.com
66.90.103.37    proxylobby.info
64.151.124.5    123download.info
                123proxygod.info
                antiwork.info
                banover.info
                beanon.info
                beanybody.info
                beanyone.info
                beanywhere.info
                bestsupersearchsite.com
                bestwebsitefilter.com
                besurfing.com
                bewherever.info
                breaknow.info
                breakrules.info
                burnoutproxy.info
                cancerproxy.info
                cantstopit.info
                cantstopthis.info
                classsucks.info
                cutchains.info
                earlylunch.info
                facebookfilter.info
                filtermyspace.info
                finallyonline.info
                financialproxy.info
                geteverywhere.info
                getsomewhere.info
                getthrough.info
                goeverywhere.info
                gosomewhere.info
                hatework.info
                hideguy.info
                hidemyinternet.com
                imfreenow.info
                instantinvisible.info
                leetproxy.info
                mylinkladder.com
                mysitefilter.info
                myspacefilter.info
                myspacekid.info
                myspaceproxy123.info
                mysupersearchsite.com
                mywebsitefilter.com
                nerdproxy.info
                newsupersearchsite.com
                newwebsitefilter.com
                notbound.info
                opentobrowse.com
                secondperiod.info
                sitefilter.info
                sneakyinternet.com
                sneakysurfer.info
                soccerproxy.info
                sportsproxy.info
                springproxy.info
                stealthysurfers.com
                studentloanproxy.info
                summerproxy.info
                themyspacebypass.info
                themyspacekid.info
                theproxybypass.info
                theproxygod.info
                theproxyhost.com
                thesitefilter.info
                thesneakysurfer.info
                thesupersearchsite.com
                thewebsitefilter.com
                unblockmyspace123.info
                unblockwhat.info
66.90.104.77    proxyprince.com
64.85.160.59    surfindark.com
81.176.236.12   serjwizard.info

Code (markup):

Please, if you have the time add more proxies to this list which you have seen showing up in Google, Yahoo or MSN with cached copies of other peoples web sites.

 

Thanks for the information SilkySmooth
Really helpful

 

Thanks Pete.... I also forgot to mention that this list of IP's can also be used with !ask!'s phpLD script based solution.

 

Thanx Silky! Will try this up!

 

Thank You

I respect and hold in high regard members who are helpful and provide information to help the directory community

 

So we paste these 3 only or all those you mentioned in another box. ?

Thanks. Nice work again

 

The code given was just a short example, use all of the IP's found in the main list. I posted it seperately so as a group we can continue to add IP / Domain's to it.

 

Lol, I noticed one of my proxies is on the list.
I think this is a semi-smart move, but there are a million (at least) proxies out there.

 

Well they'll be never a way to block them ALL, but its always better to block some than none.

I was looking for something like this Silky. Thanks a whole lot for doing this. A tool like this is not only helpful to directories, but any other site you may run as well. Gj.

 

bobby, Prevention is better than cure, I am only interested in blocking proxies which are coming up with cached versions in the search engines, not out to black list all proxies.

Leopard, your quite welcome

 

LMAO .... seeing this thread i made a small hack this afternoon!
and guess what i got banned!

first time im thankful for getting banned
http://bhomiyo.com/en.xliterate/dir.vc

was a simple javascript at work, some strip javascript....so wont work there

i ran a for loop with all the names above with the function gethostbyname();
following are the IPs i get,

(assumming this is the way to go about it!, never played with IPs b4)

 

I'm getting the same message from them and all I have done is block their IP Address.

http://bhomiyo.com/en.xliterate/directoryshare.com

 

i made a small php code, similiar to ban IPs,
putting a small js function to redirect to my site!

anyways, anything that works,

Thanx again Silky!

M.

 

Great post and just to add to this here's an additional snippet for you to consider....

Full post here http://www.seofaststart.com/blog/google-proxy-hacking

Hope it helps.

 

Ooops, thought I'd put that link in already

And I agree, sadly though it's like Google telling people to use nofollow on paid links, it's something thats never going to catch on.

 

lmao; yep its a never ending story mate.

 

Thanks SilkySmooth!!!

As already mentioned there is no end to block all proxies, but it's better than nothing.

I guess Google should be worry about this issue and start working on it.

 

Here's another list with some more IPs:
http://forums.digitalpoint.com/showthread.php?p=4698107

 

It's really sad anyone has to implement such protection. Why the fonk is Google not fixing the problem with proxies and SERPs?

 

Thanks for the link and list of IP's gostats, rep sent