how to get rid of unwanted bots? white list, black list? how to do it?

Discussion in 'WordPress' started by draligus, Jan 1, 2013.

0
  1. #1
    Unknown robot (identified by 'bot*') 71,676+192
    Unknown robot (identified by 'spider') 2,042+103
    Unknown robot (identified by '*bot') 1,559+221
    Unknown robot (identified by 'robot') 1,751+23
    Unknown robot (identified by 'crawl') 860+107

    how to get rid of unwanted bots? white list, black list? how to do it?
     
    draligus, Jan 1, 2013 IP
  2. rakesh kumar

    rakesh kumar Active Member

    Messages:
    393
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    73
    #2
    block all the unwanted bots using robots.txt file.
     
    rakesh kumar, Jan 1, 2013 IP
  3. draligus

    draligus Well-Known Member

    Messages:
    389
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    110
    #3
    thaks but how?
     
    draligus, Jan 1, 2013 IP
  4. slimrider94

    slimrider94 Member

    Messages:
    104
    Likes Received:
    2
    Best Answers:
    0
    Trophy Points:
    28
    #4
    In your robots.txt file you can choose to show the file only to certain search engines. If you only want google to see it then use this code:
    User-Agent: googlebot
    Disallow: /
     
    slimrider94, Jan 1, 2013 IP
  5. ryan_uk

    ryan_uk Illustrious Member

    Messages:
    3,983
    Likes Received:
    1,022
    Best Answers:
    33
    Trophy Points:
    465
    #5
    ryan_uk, Jan 2, 2013 IP