CAn i have a code in robot.txt, or .htaccess that could stop a speciafic user agent

Discussion in 'Apache' started by salar, Mar 20, 2006.

  1. #1
    Top 15 of 295 Total User Agents

    # Hits User Agent

    1 139298 19.76% Mozilla/5.0 (Windows; U; Windows NT 5.1; en- US; rv:1.4b) Geck
    2 122064 17.31% FUCKOFF
    3 105017 14.89% fuckyou
    4 83123 11.79% KUSMADAR
    5 82810 11.74% FUCKYOU MADARCHOD
    6 41090 5.83% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
    7 23508 3.33% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET
    8 13778 1.95% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
    9 13405 1.90% FUCKYOU
    10 6544 0.93% Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
     
    salar, Mar 20, 2006 IP
  2. salar

    salar Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I am really fed up with DDOS attacks, i get attacks nearly everyday on my site, and when i checked the statistics i found out that the attackers used some specific user agents to attack, the names were usually abuses, like i have pasted the user agent statistics in the above thread, with number of hits and name of the user agent.


    I want help regarding this problem

    i am running Xoops (a CMS) on a shared hosting server. therefore after every attck i have to close my site, for many days.

    I will be waiting for your replies
     
    salar, Mar 20, 2006 IP
  3. hans

    hans Well-Known Member

    Messages:
    2,923
    Likes Received:
    126
    Best Answers:
    1
    Trophy Points:
    173
    #3
    check the IP used by those agents
    if you have but a few precise IPs used - then an entry in .htaccess like

    deny from 71.14.26.84
    deny from 71.14.26.85

    ( replaced by your abusers IP 1 and 2 or more line for more IPs ) would do the job

    i have a similar problem with one single agent but many different and apparently changîng IPs and have no solution yet
     
    hans, Mar 20, 2006 IP
  4. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #4
    This might work in .htaccess:

    SetEnvIf user-agent "^KUSMADAR" DDoSer
    Order Allow,deny
    Allow from all
    deny from env=DDoSer

    But I've never tested such a thing so I can't guarantee it works.

    EDIT: env=DDoSer
     
    ryan_uk, Mar 20, 2006 IP
  5. ServerUnion

    ServerUnion Peon

    Messages:
    3,611
    Likes Received:
    296
    Best Answers:
    0
    Trophy Points:
    0
    #5
    You may want to look into the IP's they are using and limit the access VIA .htaccess. robots.txt is just a suggestion for the robots and will not force the disallow.

    You may want to also think about who you are pissing off and stop that activity.

    Good luck...
     
    ServerUnion, Mar 20, 2006 IP
  6. hans

    hans Well-Known Member

    Messages:
    2,923
    Likes Received:
    126
    Best Answers:
    1
    Trophy Points:
    173
    #6
    ryan_uk

    according to
    http://httpd.apache.org/docs/2.2/howto/access.html

    your suggestion should work
    i just trest it on my domain as i have only one bot to exclude

    the original posted solution did not work
    now with the apache.org solution it seems also NOT to work
    i tried with or withotu the quotes ... and the bot still gets a 200 response
     
    hans, Mar 20, 2006 IP
  7. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #7
    remove the ^
     
    ryan_uk, Mar 20, 2006 IP
  8. hans

    hans Well-Known Member

    Messages:
    2,923
    Likes Received:
    126
    Best Answers:
    1
    Trophy Points:
    173
    #8
    :)
    now he gets a 403

    the complete lines in .htaccess now are:

    --------
    SetEnvIf user-agent "Indy Library" stayout=1
    Order Allow,deny
    Allow from all
    deny from env=stayout
    --------

    thanks for your spontaneous help
     
    hans, Mar 20, 2006 IP
  9. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #9
    I forgot to mention earlier, bad-behaviour might be worth checking out:
    http://www.ioerror.us/software/bad-behavior/

    It's a php solution. I use this on one of my sites (that uses drupal cms). It's a very nice script for blocking DDoS and spam.
     
    ryan_uk, Mar 20, 2006 IP
  10. salar

    salar Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #10
    it worked gr8, but is their any way to block more then one such nasty user agents/bots. this code block one browser at a given time.
     
    salar, Mar 23, 2006 IP
  11. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #11
    ryan_uk, Mar 23, 2006 IP
  12. salar

    salar Peon

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #12
    Mozilla/4.0 (compatible; MSIE 5.13; Mac_PowerPC)

    i tried my level best to block this user agent but it is continously used to disrupt us... it is not stopping from the above provided code.
     
    salar, Mar 25, 2006 IP