Is this robots.txt file correct?

Discussion in 'robots.txt' started by Imozeb, May 5, 2010.

  1. #1
    This is my directory layout:

    www.mysite.com --> html --> file1.html
    www.mysite.com --> html --> file2.html
    www.mysite.com --> html --> file3.html

    www.mysite.com --> images--> img1.gif
    www.mysite.com --> images--> img2.gif
    www.mysite.com --> images--> img3.gif
    www.mysite.com --> button --> button1.gif
    www.mysite.com --> button --> button2.gif

    How would I rescrict access from all bots to (file3.html, and the images folder, and the button folder)? I've already tried something but I want to make sure it is right.

    Thanks.
     
    Imozeb, May 5, 2010 IP
  2. Imozeb

    Imozeb Peon

    Messages:
    666
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #2
    Would this work?

    Robots.txt
    
    User-agent: *
    Disallow: /images/
    User-agent: *
    Disallow: /button/
    User-agent: *
    Disallow: /html/file3.html
    
    Code (markup):
     
    Imozeb, May 6, 2010 IP
  3. Imozeb

    Imozeb Peon

    Messages:
    666
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #3
    Is this really a difficult question bc no one is answering it... :(
     
    Imozeb, May 7, 2010 IP
  4. Snoork Hosting

    Snoork Hosting Active Member

    Messages:
    117
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    68
    #4
    Yes, this will work.

    There is an official robots.txt file generate in Google Analytics which allows you to generate your own robots.txt file in a user friendly interface.
     
    Snoork Hosting, May 7, 2010 IP
  5. Imozeb

    Imozeb Peon

    Messages:
    666
    Likes Received:
    3
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Okay. Thank you for the info.
     
    Imozeb, May 8, 2010 IP
  6. bodiesbeautifuluk

    bodiesbeautifuluk Peon

    Messages:
    5
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #6
    This robots.txt file will work successfully.No need to worry about it.

    James
     
    bodiesbeautifuluk, May 12, 2010 IP
  7. onlinesujeet

    onlinesujeet Greenhorn

    Messages:
    49
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    16
    #7
    robots format is:
    User-agent: *
    Disallow: /images/
    Disallow: /button/
    Disallow: /html/file3.html
     
    onlinesujeet, May 18, 2010 IP
  8. kevoh99

    kevoh99 Peon

    Messages:
    175
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #8
    I like this one as opposed to the one above. They are both ok anyway
     
    kevoh99, May 25, 2010 IP
  9. naanmurugan

    naanmurugan Guest

    Messages:
    4
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #9
    This is the general format of robots.txt:
    User-agent: *
    Disallow: /images/
    Disallow: /button/
    Disallow: /html/file3.html
     
    naanmurugan, May 29, 2010 IP
  10. lonew

    lonew Peon

    Messages:
    131
    Likes Received:
    1
    Best Answers:
    0
    Trophy Points:
    0
    #10
    Yes it is correct.
     
    lonew, May 29, 2010 IP
  11. luckzz1000

    luckzz1000 Banned

    Messages:
    693
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    51
    #11
    thanks for share info
     
    luckzz1000, May 30, 2010 IP