? abouta bot

Discussion in 'robots.txt' started by templates, Feb 23, 2008.

  1. #1
    i do not have a robots.txtr file on one of my sites,but i need one.this bot has used 4.1 gigs of BW this month!..its
    Unknown robot (identified by 'bot/' or 'bot-') 11579 4.17 GB 23 Feb 2008 - 18:07
    how should the text file be setup to allow all bots/spiders but this one?and do i just upload to my root?thanks
     
    templates, Feb 23, 2008 IP
  2. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #2
    User-agent: bot/ or bot-(as per the bot name)
    Disallow: /

    User-agent: *
    Disallow: /your private file
     
    manish.chauhan, Apr 4, 2008 IP
  3. manish.chauhan

    manish.chauhan Well-Known Member

    Messages:
    1,682
    Likes Received:
    35
    Best Answers:
    0
    Trophy Points:
    110
    #3
    Once you have made your robots.txt file,just upload it in your root directory on the server..
     
    manish.chauhan, Apr 4, 2008 IP