Question Correct way to block bad bots in httpd.conf?

Discussion in 'Apache' started by kokopelli, Sep 29, 2011.

  1. #1
    I want to block some bad bots and hosts via httpd.conf and have tried adding the following to httpd.conf via WHM > Apache Configuration >Include Editor (first by adding it to "Pre Main Include", then to "Pre VirtualHost Include", then to "Post VirtualHost Include"), to no avail. [After each case, I restarted Apache]
    # START BLOCK BAD BOTS
    
    SetEnvIfNoCase User-Agent "^somespider" UnwantedRobot
    SetEnvIfNoCase User-Agent "^somebot" UnwantedRobot
    SetEnvIfNoCase User-Agent "^HTTrack" UnwantedRobot
    
    <Directory />
        Order Allow,Deny
        Allow from all
        Deny from env=UnwantedRobot
    </Directory>
    
    # END BLOCK BAD BOTS
    Code (markup):
    The test I ran with HTTrack didn't work, it still got through. What I am doing wrong? Any help will be appreciated.

    BTW I also tried the following variations of the <Directory /> directive:

    <Directory "/home/">
        Order Allow,Deny
        Allow from all
        Deny from env=UnwantedRobot
    </Directory>
    <Directory "/var/www/">
        Order Allow,Deny
        Allow from all
        Deny from env=UnwantedRobot
    </Directory>
    
    Code (markup):
     
    kokopelli, Sep 29, 2011 IP