1. Advertising
    y u no do it?

    Advertising (learn more)

    Advertise virtually anything here, with CPM banner ads, CPM email ads and CPC contextual links. You can target relevant areas of the site and show ads based on geographical location of the user if you wish.

    Starts at just $1 per CPM or $0.10 per CPC.

Block all Spam bots using .htaccess

Discussion in 'Apache' started by manov, Aug 27, 2012.

  1. #1
    I only want to allow Google, MSN and Yahoo bot to crawl my site and block all others using .htacess
    Setting rules on rbobots.txt file is useless because spammers break them.
     
    manov, Aug 27, 2012 IP
  2. GMF

    GMF Well-Known Member

    Messages:
    855
    Likes Received:
    113
    Best Answers:
    19
    Trophy Points:
    145
    #2
    what? How?


    Anyways

    found something that might help you out
    http://www.javascriptkit.com/howto/htaccess13.shtml
    http://www.askapache.com/htaccess/blocking-bad-bots-and-scrapers-with-htaccess.html
    Code (markup):
     
    GMF, Aug 27, 2012 IP
  3. manov

    manov Well-Known Member

    Messages:
    151
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    105
    #3
    Thanks, but that's not what I was looking for. Instead of blocking long list of bad bots, I only want to allow good bots(like google and msn)
     
    manov, Aug 27, 2012 IP
  4. atxsurf

    atxsurf Peon

    Messages:
    2,394
    Likes Received:
    21
    Best Answers:
    1
    Trophy Points:
    0
    #4
    more correctly they don't bother to check them. Why would they, they on a mission to spam anyways.
     
    atxsurf, Aug 27, 2012 IP
  5. pankajrko

    pankajrko Peon

    Messages:
    2
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Excessive bots other than google seems useless to me
     
    pankajrko, Oct 5, 2012 IP