Stoping Referrer Spam

[kusadasi-guy]

i am hitting by wan*er referrer spammers (like you all). How can i stop them?
They are using proxy servers, so that banning thier ip addresses is not a solid idea. ANd i dont want to reduce my servers performance with putting hundreds of lines to htaccess ( deny from spammer-casino/dot/com ...etc)

i scanned my logs and if i block some keywords then it would be prevent %80 of these spammers.

For example; if referrer link contains "casino" , "pharmacy" keywords, then stop it.

How can i do that with htaccess?

[digitalpoint]

There isn't a "sure fire" way to do it unfortunately... but if you check some of the similar threads (look at the bottom of this page), there are some other threads about it.

[Smyrl]

I do not have the answer, wish I did. I have seen two methods used that have some appeal to me. Both involve using .htaccess and doing something with sites based on

• Words appearing in URL
• URLs havng two or more dashes in them.

I have lost the URL showing first method. If someone has it I would appreciate your posting the link.

Thanks,
Shannon

[kusadasi-guy]

I found it!

RewriteEngine On

SetEnvIfNoCase Referer ".*(casino).*" BadReferrer
SetEnvIfNoCase Referer ".*(pharmacy).*" BadReferrer
SetEnvIfNoCase Referer ".*(gambling).*" BadReferrer
order deny,allow
deny from env=BadReferrer

IF a domain name (or its subdomain name) contains "casino", "pharmacy" and/or "gambling" then deny it.

I can prevent %70 of all my spammers with just few words.


Should i also put these lines to httpd.conf file to prevent all of the domains in the server?

[Smyrl]

Originally Posted by kusadasi-guy

I found it!

RewriteEngine On

SetEnvIfNoCase Referer ".*(casino).*" BadReferrer
SetEnvIfNoCase Referer ".*(pharmacy).*" BadReferrer
SetEnvIfNoCase Referer ".*(gambling).*" BadReferrer
order deny,allow
deny from env=BadReferrer

IF a domain name (or its subdomain name) contains "casino", "pharmacy" and/or "gambling" then deny it.

I can prevent %70 of all my spammers with just few words.


Should i also put these lines to httpd.conf file to prevent all of the domains in the server?

Here is other code I have found.

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L]

Can I use both to try to a double barrel shotgun approach?

Shannon

[danpadams]

Smyrl, What does your code do exactally?

[iShopHQ]

If you're on a Windows box, you can set up a similar function in the global.asa file. Grab incoming referers, do an INSTR() check, and then a response.end that stops them in their tracks.

[J.D.]

Originally Posted by danpadams

Smyrl, What does your code do exactally?

It checks if the referrer string matches URLs like these:

http://www.abc-def-
abc-def-

And if it does, redirects the browser to the IP address of the browser itself.

J.D.

[danpadams]

I hate to say it, but that is sorta being downright mean. In my opinion from the way I have my personal stuff setup, it wouldn't be as mean as possible as I have a webserver on my outside address, but that would mean the bandwidth would be used (the webserver is not in my own box but on the same IP #, why not just point to http://127.0.0.1?

[Smyrl]

Originally Posted by J.D.

It checks if the referrer string matches URLs like these:

http://www.abc-def-
abc-def-

And if it does, redirects the browser to the IP address of the browser itself.

J.D.

J.D., Can I use this

RewriteEngine On

SetEnvIfNoCase Referer ".*(casino).*" BadReferrer
SetEnvIfNoCase Referer ".*(pharmacy).*" BadReferrer
SetEnvIfNoCase Referer ".*(gambling).*" BadReferrer
order deny,allow
deny from env=BadReferrer


and

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L]

both in my .htaccess?

Shannon

[J.D.]

Originally Posted by Smyrl

J.D., Can I use this

I never used SetEnvIf and can't say anything regarding its performance, but the fact that it assigns variables makes me think that it is probably not as fast as mod_rewrite.

I would rewrite the expressions as keyword instead of .*(keyword).* for performance reasons, but otherwise, either method or both should work fine.

J.D.

[kusadasi-guy]

Hello JD,

As i understand from your reply, these lines are better than SetEnvIf rules, right?


RewriteEngine On
RewriteCond %{HTTP_REFERER} (casino) [OR]
RewriteCond %{HTTP_REFERER} (pharmacy) [OR]
RewriteCond %{HTTP_REFERER} (gambling) [NC]
RewriteRule .* - [F]

[J.D.]

These rules will return 403 (forbidden) when referrer contains any of the specified words:

RewriteCond %{HTTP_REFERER} poker [OR,NC]
RewriteCond %{HTTP_REFERER} casino [NC]
RewriteRule ^.? - [F]

Edit: I think rewrite rules will work faster, but I didn't actually run a test to verify this.

J.D.

[J.D.]

Also, don't forget, the order in which rewrite rules are specified does matter. Rules denying access should go first (otherwise some of the perpetrators may slip through if they hit a preceeding rule with an [L] flag).

J.D.

[kusadasi-guy]

Seems work perfectly J.D.
Thank You so much

Here is my new htaccess;

RewriteEngine on
RewriteCond %{HTTP_REFERER} pharmacy [NC,OR]
RewriteCond %{HTTP_REFERER} viagra [NC,OR]
RewriteCond %{HTTP_REFERER} porn [NC,OR]
RewriteCond %{HTTP_REFERER} casino [NC,OR]
RewriteCond %{HTTP_REFERER} gambling [NC,OR]
RewriteCond %{HTTP_REFERER} phentermine [NC,OR]
RewriteCond %{HTTP_USER_AGENT}  ^LWP* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [NC,OR]
RewriteCond %{HTTP_USER_AGENT}  ^lwp* [NC]
RewriteRule  .*      - [F]

BTW, whats the difference between "RewriteRule ^.? - [F]" and RewriteRule .* - [F] ?

[J.D.]

Originally Posted by kusadasi-guy

RewriteCond %{HTTP_USER_AGENT} ^LWP* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^lwp* [NC]

These are the same - [NC] stands for canse-insensitive string comparison.

Originally Posted by kusadasi-guy

BTW, whats the difference between "RewriteRule ^.? - [F]" and RewriteRule .* - [F] ?

The first one will work faster because the regular expresion parser will have to match only one optional character at the beginning of the referrer string. The second one means "one or more" and the parser may need to process more characters.

J.D.

[kyle422]

Thanks for posting the solution kusudasi. I put it in my .htaccess and it works well for me.

[kusadasi-guy]

Thanks to JD actually, i learned from him.

Just last thing, how can i do it for my server's websites? Should i add that lines (except "RewriteEngine on") to my httpd.conf file? or add to another file?

[J.D.]

Originally Posted by kusadasi-guy

Thanks to JD actually, i learned from him.

Just last thing, how can i do it for my server's websites? Should i add that lines (except "RewriteEngine on") to my httpd.conf file? or add to another file?

Rewrite rules can be placed in the server config, virtual domain sections or in the .htaccess file. Your choice.

J.D.

[TwisterMc]

Any chance this is for a wordpress blog? I just implemented a plugin that helps block referrer spam.