1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Stoping Referrer Spam

Discussion in 'Traffic Analysis' started by kusadasi-guy, Mar 24, 2005.

  1. #1
    i am hitting by wan*er referrer spammers (like you all). How can i stop them?
    They are using proxy servers, so that banning thier ip addresses is not a solid idea. ANd i dont want to reduce my servers performance with putting hundreds of lines to htaccess ( deny from spammer-casino/dot/com ...etc)

    i scanned my logs and if i block some keywords then it would be prevent %80 of these spammers.

    For example; if referrer link contains "casino" , "pharmacy" keywords, then stop it.
    SEMrush
    How can i do that with htaccess?
     
    kusadasi-guy, Mar 24, 2005 IP
    SEMrush
  2. digitalpoint

    digitalpoint Overlord of no one Staff

    Messages:
    38,198
    Likes Received:
    2,584
    Best Answers:
    457
    Trophy Points:
    710
    Digital Goods:
    29
    #2
    There isn't a "sure fire" way to do it unfortunately... but if you check some of the similar threads (look at the bottom of this page), there are some other threads about it.
     
    digitalpoint, Mar 24, 2005 IP
  3. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,406
    Likes Received:
    1,486
    Best Answers:
    76
    Trophy Points:
    510
    #3
    I do not have the answer, wish I did. I have seen two methods used that have some appeal to me. Both involve using .htaccess and doing something with sites based on

    • Words appearing in URL
    • URLs havng two or more dashes in them.

    I have lost the URL showing first method. If someone has it I would appreciate your posting the link.

    Thanks,
    Shannon
     
    Smyrl, Mar 24, 2005 IP
  4. kusadasi-guy

    kusadasi-guy Peon

    Messages:
    83
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #4
    I found it!



    RewriteEngine On
    
    SetEnvIfNoCase Referer ".*(casino).*" BadReferrer
    SetEnvIfNoCase Referer ".*(pharmacy).*" BadReferrer
    SetEnvIfNoCase Referer ".*(gambling).*" BadReferrer
    order deny,allow
    deny from env=BadReferrer
    PHP:
    IF a domain name (or its subdomain name) contains "casino", "pharmacy" and/or "gambling" then deny it.

    I can prevent %70 of all my spammers with just few words.


    Should i also put these lines to httpd.conf file to prevent all of the domains in the server?
     
    kusadasi-guy, Mar 24, 2005 IP
    Smyrl likes this.
  5. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,406
    Likes Received:
    1,486
    Best Answers:
    76
    Trophy Points:
    510
    #5
    Here is other code I have found.

    RewriteEngine On
    RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
    RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L]

    Can I use both to try to a double barrel shotgun approach?

    Shannon
     
    Smyrl, Mar 24, 2005 IP
  6. danpadams

    danpadams Peon

    Messages:
    101
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #6
    Smyrl, What does your code do exactally?
     
    danpadams, Mar 25, 2005 IP
  7. iShopHQ

    iShopHQ Peon

    Messages:
    644
    Likes Received:
    33
    Best Answers:
    0
    Trophy Points:
    0
    #7
    If you're on a Windows box, you can set up a similar function in the global.asa file. Grab incoming referers, do an INSTR() check, and then a response.end that stops them in their tracks.
     
    iShopHQ, Mar 25, 2005 IP
  8. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #8
    It checks if the referrer string matches URLs like these:

    http://www.abc-def-
    abc-def-

    And if it does, redirects the browser to the IP address of the browser itself.

    J.D.
     
    J.D., Mar 25, 2005 IP
  9. danpadams

    danpadams Peon

    Messages:
    101
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #9
    I hate to say it, but that is sorta being downright mean. In my opinion from the way I have my personal stuff setup, it wouldn't be as mean as possible as I have a webserver on my outside address, but that would mean the bandwidth would be used (the webserver is not in my own box but on the same IP #, why not just point to http://127.0.0.1?
     
    danpadams, Mar 25, 2005 IP
  10. Smyrl

    Smyrl Tomato Republic Staff

    Messages:
    13,406
    Likes Received:
    1,486
    Best Answers:
    76
    Trophy Points:
    510
    #10
    J.D., Can I use this

    RewriteEngine On

    SetEnvIfNoCase Referer ".*(casino).*" BadReferrer
    SetEnvIfNoCase Referer ".*(pharmacy).*" BadReferrer
    SetEnvIfNoCase Referer ".*(gambling).*" BadReferrer
    order deny,allow
    deny from env=BadReferrer


    and

    RewriteEngine On
    RewriteCond %{HTTP_REFERER} ^(http://www.)[a-z]+-[a-z]+- [NC]
    RewriteRule ^(.*) http://%{REMOTE_ADDR}/ [R=301,L]

    both in my .htaccess?

    Shannon
     
    Smyrl, Mar 25, 2005 IP
  11. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #11
    I never used SetEnvIf and can't say anything regarding its performance, but the fact that it assigns variables makes me think that it is probably not as fast as mod_rewrite.

    I would rewrite the expressions as keyword instead of .*(keyword).* for performance reasons, but otherwise, either method or both should work fine.

    J.D.
     
    J.D., Mar 25, 2005 IP
  12. kusadasi-guy

    kusadasi-guy Peon

    Messages:
    83
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Hello JD,

    As i understand from your reply, these lines are better than SetEnvIf rules, right?


    RewriteEngine On
    RewriteCond %{HTTP_REFERER} (casino) [OR]
    RewriteCond %{HTTP_REFERER} (pharmacy) [OR]
    RewriteCond %{HTTP_REFERER} (gambling) [NC]
    RewriteRule .* - [F]
     
    kusadasi-guy, Mar 25, 2005 IP
  13. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #13
    These rules will return 403 (forbidden) when referrer contains any of the specified words:

    RewriteCond %{HTTP_REFERER} poker [OR,NC]
    RewriteCond %{HTTP_REFERER} casino [NC]
    RewriteRule ^.? - [F]

    Edit: I think rewrite rules will work faster, but I didn't actually run a test to verify this.

    J.D.
     
    J.D., Mar 25, 2005 IP
  14. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #14
    Also, don't forget, the order in which rewrite rules are specified does matter. Rules denying access should go first (otherwise some of the perpetrators may slip through if they hit a preceeding rule with an [L] flag).

    J.D.
     
    J.D., Mar 25, 2005 IP
  15. kusadasi-guy

    kusadasi-guy Peon

    Messages:
    83
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #15
    Seems work perfectly J.D.
    Thank You so much

    Here is my new htaccess;
    RewriteEngine on
    RewriteCond %{HTTP_REFERER} pharmacy [NC,OR]
    RewriteCond %{HTTP_REFERER} viagra [NC,OR]
    RewriteCond %{HTTP_REFERER} porn [NC,OR]
    RewriteCond %{HTTP_REFERER} casino [NC,OR]
    RewriteCond %{HTTP_REFERER} gambling [NC,OR]
    RewriteCond %{HTTP_REFERER} phentermine [NC,OR]
    RewriteCond %{HTTP_USER_AGENT}  ^LWP* [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Teleport [NC,OR]
    RewriteCond %{HTTP_USER_AGENT} ^Wget [NC,OR]
    RewriteCond %{HTTP_USER_AGENT}  ^lwp* [NC]
    RewriteRule  .*      - [F]
    PHP:
    BTW, whats the difference between "RewriteRule ^.? - [F]" and RewriteRule .* - [F] ?
     
    kusadasi-guy, Mar 25, 2005 IP
  16. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #16
    These are the same - [NC] stands for canse-insensitive string comparison.

    The first one will work faster because the regular expresion parser will have to match only one optional character at the beginning of the referrer string. The second one means "one or more" and the parser may need to process more characters.

    J.D.
     
    J.D., Mar 25, 2005 IP
  17. kyle422

    kyle422 Peon

    Messages:
    290
    Likes Received:
    8
    Best Answers:
    0
    Trophy Points:
    0
    #17
    Thanks for posting the solution kusudasi. I put it in my .htaccess and it works well for me. :)
     
    kyle422, Mar 25, 2005 IP
  18. kusadasi-guy

    kusadasi-guy Peon

    Messages:
    83
    Likes Received:
    4
    Best Answers:
    0
    Trophy Points:
    0
    #18
    Thanks to JD actually, i learned from him.

    Just last thing, how can i do it for my server's websites? Should i add that lines (except "RewriteEngine on") to my httpd.conf file? or add to another file?
     
    kusadasi-guy, Mar 25, 2005 IP
  19. J.D.

    J.D. Peon

    Messages:
    1,198
    Likes Received:
    64
    Best Answers:
    0
    Trophy Points:
    0
    #19
    Rewrite rules can be placed in the server config, virtual domain sections or in the .htaccess file. Your choice.

    J.D.
     
    J.D., Mar 25, 2005 IP
  20. TwisterMc

    TwisterMc Mac Guru

    Messages:
    972
    Likes Received:
    13
    Best Answers:
    0
    Trophy Points:
    0
    #20
    Any chance this is for a wordpress blog? I just implemented a plugin that helps block referrer spam.
     
    TwisterMc, Mar 25, 2005 IP